Data Engineer

Description

We are looking for an experienced Data Engineer to join our team in Jacksonville, Florida. In this role, you will take the lead in designing and building a cutting-edge Azure lakehouse platform that enables business leaders to access analytics through natural language queries. This position combines hands-on technical expertise with leadership responsibilities, offering an opportunity to mentor a team of skilled engineers while driving innovation.

Responsibilities:
Architect and develop a robust Azure lakehouse platform, utilizing Azure Data Lake Gen2, Delta Lake, and PySpark to create efficient data pipelines.
Implement a semantic layer and metric store to ensure consistent data translation and definitions across the organization.
Design and maintain real-time and batch data pipelines, incorporating medallion architecture, schema evolution, and data contracts.
Build retrieval systems for large language models (LLMs) using Azure OpenAI and vectorized Delta tables to support chat-based analytics.
Ensure data quality, lineage, and observability through tools like Great Expectations and Unity Catalog, while optimizing costs through partitioning and compaction.
Develop automated systems for anomaly detection and alerting using Azure ML pipelines and Event Grid.
Collaborate with product and operations teams to translate complex business questions into actionable data models and queries.
Lead and mentor a team of data and Python engineers, establishing best practices in CI/CD, code reviews, and documentation.
Ensure compliance with security, privacy, and governance standards by designing and implementing robust data handling protocols.

Requirements

Minimum of 7-10 years of experience in data engineering, with at least 2-4 years leading technical teams.
Expertise in Azure Data Lake Gen2, Delta Lake, Unity Catalog, PySpark, and Python.
Proven ability to design and implement Lakehouse architectures, including medallion frameworks and schema evolution.
Proficiency in Databricks workflows, query optimization, and monitoring tools.
Experience with semantic/metrics layers and advanced analytics for business applications.
Familiarity with Azure OpenAI integrations, vector search, and structured data grounding.
Strong skills in CI/CD development using GitHub Actions or Azure DevOps, as well as infrastructure automation through Bicep or Terraform.
Excellent communication skills, with the ability to mentor engineers and translate business needs into technical solutions.

Technology Doesn't Change the World, People Do.

Robert Half is the world's first and largest specialized talent solutions firm that connects highly qualified job seekers to opportunities at great companies. We offer contract, temporary and permanent placement solutions for finance and accounting, technology, marketing and creative, legal, and administrative and customer support roles.

Robert Half works to put you in the best position to succeed. We provide access to top jobs, competitive compensation and benefits, and free online training. Stay on top of every opportunity - whenever you choose - even on the go. Download the Robert Half app and get 1-tap apply, notifications of AI-matched jobs, and much more.

All applicants applying for U.S. job openings must be legally authorized to work in the United States. Benefits are available to contract/temporary professionals, including medical, vision, dental, and life and disability insurance. Hired contract/temporary professionals are also eligible to enroll in our company 401(k) plan. Visit roberthalf.gobenefits.net for more information.

2025 Robert Half. An Equal Opportunity Employer. M/F/Disability/Veterans. By clicking "Apply Now," you're agreeing to Robert Half's Terms of Use and Privacy Notice.