Description The Data Engineer role focuses on designing, building, and optimizing scalable data solutions that support diverse business needs. This position requires the ability to work independently while collaborating effectively in a fast-paced, agile environment. The individual in this role partners with cross-functional teams to gather data requirements, recommend enhancements to existing data pipelines and architectures, and ensure the reliability, performance, and efficiency of data processes.
Responsibilities
- Support the team's adoption and continued evolution of the Databricks platform, leveraging features such as Delta Live Tables, workflows, and related tooling
- Design, develop, and maintain data pipelines that extract data from relational sources, load it into a data lake, transform it as needed, and publish it to a Databricks-based lakehouse environment
- Optimize data pipelines and processing workflows to improve performance, scalability, and overall efficiency
- Implement data quality checks and validation logic to ensure data accuracy, consistency, and completeness
- Create and maintain documentation including data mappings, data definitions, architectural diagrams, and data flow diagrams
- Develop proof-of-concepts to evaluate and validate new technologies, tools, or data processes
- Deploy, manage, and support code across non-production and production environments
- Investigate, troubleshoot, and resolve data-related issues, including identifying root causes and implementing fixes
- Identify performance bottlenecks and recommend optimization strategies, including database tuning and query performance improvements
Requirements Qualifications
- Bachelor's degree in Computer Science, Data Science, Software Engineering, Information Systems, or a related quantitative discipline
- 4+ years of experience in data-focused roles such as Data Engineer, ETL Engineer, Data Architect, or similar
- Active Databricks Data Engineer or Analyst certification
Technical Skills & Experience
- 4+ years of hands-on experience developing in Python
- 3+ years of experience working with Databricks, including production implementations involving data structures, data storage, change data capture, pipeline optimization, and best practices
- 3+ years of experience with Kimball dimensional modeling, including star schemas, fact tables, Type 1 and Type 2 dimensions, aggregates, and a strong understanding of ELT/ETL methodologies
- 3+ years of experience writing complex SQL and PL/SQL
- 2+ years of experience using Airflow for workflow orchestration
- 3+ years of experience working with relational databases (Oracle or similar platforms)
- 2+ years of experience with NoSQL databases such as MongoDB, Cosmos DB, DocumentDB, or comparable technologies
- 2+ years of cloud platform experience (Azure or equivalent)
- Experience implementing CI/CD practices using Git-based workflows and DevOps tools
- Familiarity with modern data storage formats including Parquet, Arrow, and Avro
Technology Doesn't Change the World, People Do.
Robert Half is the world's first and largest specialized talent solutions firm that connects highly qualified job seekers to opportunities at great companies. We offer contract, temporary and permanent placement solutions for finance and accounting, technology, marketing and creative, legal, and administrative and customer support roles.
Robert Half works to put you in the best position to succeed. We provide access to top jobs, competitive compensation and benefits, and free online training. Stay on top of every opportunity - whenever you choose - even on the go. Download the Robert Half app and get 1-tap apply, notifications of AI-matched jobs, and much more.
All applicants applying for U.S. job openings must be legally authorized to work in the United States. Benefits are available to contract/temporary professionals, including medical, vision, dental, and life and disability insurance. Hired contract/temporary professionals are also eligible to enroll in our company 401(k) plan. Visit roberthalf.gobenefits.net for more information.
2025 Robert Half. An Equal Opportunity Employer. M/F/Disability/Veterans. By clicking "Apply Now," you're agreeing to Robert Half's Terms of Use and Privacy Notice.
Never miss an opportunity! Create an alert based on the job you applied for.