Azure Data Engineer with Databricks & medallion architecture

Data Engineer

Role Summary

As a Data Engineer on the team, you will design, build, and operate high-quality data pipelines and platforms that power advanced analytics, optimization models, and enterprise dashboards. You will work at the intersection of data engineering, analytics engineering, and platform enablement—supporting everything from proof-of-concept workflows to production-grade, high-SLA pipelines running on Databricks.

Key Responsibilities

• Design, build, and maintain scalable data pipelines for ingestion, transformation, and serving of analytics-ready datasets across multiple regions and business domains.
• Partner with data scientists, optimization engineers, and business stakeholders to translate analytical requirements into clean, reusable data models and curated datasets.
• Productionalize analytics and model workflows by implementing robust, monitored, and well-governed pipelines that meet enterprise standards for reliability and performance.
• Support end-to-end lifecycle execution, from MVP through production and run-and-maintain, including refactoring legacy pipelines and enabling new use cases.
• Implement best practices in data quality, testing, versioning, and CI/CD for data pipelines and analytics workflows.
• Enable platform capabilities on cloud-native analytics stacks (e.g., Databricks, Unity Catalog), including environment setup, access control, and governance.
• Troubleshoot data issues, perform root-cause analysis, and continuously improve pipeline performance and maintainability.
• Collaborate cross-functionally with IT, platform teams, and global business partners to integrate solutions with enterprise systems (e.g., planning, supply chain, commercial tools).

Required Qualifications

• Bachelor’s degree in Computer Science, Engineering, Data Science, or a related quantitative field, with 3+ years of experience in data engineering.
• Strong experience building and maintaining data pipelines using Python and SQL in a cloud environment.
• Hands-on experience with modern data platforms, particularly Databricks.
• Solid understanding of data modeling, ETL/ELT patterns, and analytics engineering concepts.
• Experience supporting production workloads with a focus on reliability, observability, and data quality.
• Ability to work effectively with both technical and non-technical stakeholders in a fast-paced, cross-functional environment.

Preferred Qualifications

• Knowledge of modern data architectures, including medallion architecture and lakehouse patterns.
• Experience with MLOps or analytics CI/CD practices.
• Experience migrating or refactoring pipelines from legacy platforms to cloud-native architectures.
• Exposure to enterprise data governance, security, and compliance requirements.
• Background in agriculture, manufacturing, or large-scale operational analytics is a plus.