Software Engineer - Datalake
Dremio
Be Part of Building the Future
Dremio is the unified lakehouse platform for self-service analytics and AI, serving hundreds of global enterprises, including Maersk, Amazon, Regeneron, NetApp, and S&P Global. Customers rely on Dremio for cloud, hybrid, and on-prem lakehouses to power their data mesh, data warehouse migration, data virtualization, and unified data access use cases. Based on open source technologies, including Apache Iceberg and Apache Arrow, Dremio provides an open lakehouse architecture enabling the fastest time to insight and platform flexibility at a fraction of the cost. Learn more at www.dremio.com.
About the role
We are looking for an experienced Software Engineer to enhance Dremio’s data warehouse capabilities on top of the datalakes across all major table/file formats and object stores. These capability advancements will increase our competitive position in the market and enable Dremio adoption for a larger set of customers.
What you’ll be doing
- Develop core components for Dremio’s query engine
- Deliver key features and feature enhancements for our customers in the Datalake like DML operations, time travel, schema evolution along with performance and reliability improvements
- Work with open source projects like Apache Iceberg, Parquet, Arrow and Calcite
- Own design, implementation, testing, and support of next-generation features related to scalability, reliability, robustness, and performance of the product
- Collaborate with Product Management to innovate and deliver on customer requirements and with Support and field teams to ensure customer success
- Understand and reason about concurrency and parallelization to deliver scalability and performance in a multithreaded and distributed environment
- Solve complex technical problems and customer issues while improving our telemetry and instrumentation to proactively detect issues before they arise and make debugging more efficient
- Work with engineering leaders to establish solid designs/architecture for upcoming features.
- Develop the future leaders of Dremio by providing continuous mentorship and coaching of junior software engineers, help with hiring and onboarding
What we’re looking for
- 4+ year of industry experience
- B.S./M.S/Equivalent in Computer Science or a related technical field or equivalent experience
- Fluency in Java, C++ or another modern language
- Strong database fundamentals including SQL, performance, and schema design and background in large scale data processing systems (e.g., Hadoop, Spark, etc.)
- Understanding of distributed file systems such as S3, ADLS, or HDFS
- Experience with Apache Iceberg, Parquet, AVRO and/or Delta Lake
- Experience with Hive and AWS Glue
- Ability to solve ambiguous, unexplored, and cross-team problems effectively
- Interested and motivated to be part of a fast-moving startup with a fun and accomplished team
- Big picture thinking, ability to scope and plan solutions for big problems and mentor others on the same
Bonus points if you have
- Hands-on experience with distributed query engines, query processing or optimization, distributed systems, concurrency control, data replication, code generation, or storage systems
- Hands on experience with AWS, Azure and Google Cloud Platform
What we value
At Dremio, we hold ourselves to high standards when it comes to People, Thinking, and Action. Our Gnarlies (that's what we call our employees) communicate with clarity, drive accountability, and are respectful towards each other. We confront brutal facts and focus on results while operating with a sense of urgency and building a "flywheel". People who like to jump in and drive momentum will thrive in our #GnarlyLife.
Dremio is an equal opportunity employer supporting workforce diversity. We do not discriminate on the basis of race, religion, color, national origin, gender identity, sexual orientation, age, marital status, protected veteran status, disability status, or any other unlawful factor.
Dremio is committed to providing any necessary accommodations for individuals with disabilities within our application and interview process. To request accommodation due to a disability, please inform your recruiter.
Dremio has policies in place to protect the personal information that employees and applicants disclose to us. Please click here to review the privacy notice.
Important Security Notice for Candidates
At Dremio, we uphold trust and transparency as paramount values in all our interactions with customers, partners, employees, and the general public. We have been targeted by individuals creating fake domains similar to ours to scam prospects and candidates. Please note that all official communications from us will be from an @dremio.com domain. If you suspect you've been targeted by a scam, it's imperative to report the incident to your local law enforcement agencies. For more information about this type of scam, please refer to Dremio's official statement here.