Job Summary: We are seeking an experienced Azure Data Tech Lead to design and deliver scalable, high-performing data solutions within the Azure ecosystem. This role involves leading end-to-end data engineering initiatives, driving architectural decisions, and building robust data pipelines using modern lakehouse architectures. The ideal candidate will have strong expertise in Databricks, Spark, data modeling, and both batch and real-time data processing. Key Responsibilities: Architect, design, and implement scalable data platforms and pipelines on Azure and Databricks. Build and optimize data ingestion, transformation, and processing workflows across batch and real-time data streams. Work extensively with ADLS, Delta Lake, and Spark (Python) for large-scale data engineering. Lead development of complex ETL/ELT pipelines ensuring quality, reliability, and performance. Design and implement data models including conceptual, logical, and physical models. Work with relational and lakehouse systems including PostgreSQL and Delta Lake. Define and enforce best practices in data governance, data quality, security, and architecture. Collaborate with cross-functional teams to translate business requirements into technical solutions. Troubleshoot production issues, optimize performance, and drive continuous improvement. Mentor junior engineers and contribute to engineering standards and reusable components. Required Qualifications: 10+ years of hands-on data engineering experience in enterprise environments. Strong expertise in Azure services, including Azure Databricks and related data services. Advanced proficiency in Apache Spark with Python (PySpark). Strong command of SQL, including query optimization and performance tuning. Deep understanding of ETL/ELT methodologies and data pipeline orchestration. Hands-on experience with Delta Lake, including ACID transactions and schema evolution. Strong experience in data modeling (normalized, dimensional, and lakehouse models). Experience with both batch and real-time/streaming data processing technologies. Strong understanding of data architecture principles, distributed systems, and cloud-native design. Ability to design end-to-end solutions and evaluate architectural trade-offs. Strong analytical, problem-solving, and communication skills. Preferred Qualifications: Experience with CI/CD tools such as Azure DevOps and Git. Familiarity with infrastructure as code tools such as Terraform or ARM templates. Exposure to data governance and cataloging tools such as Azure Purview. Experience supporting machine learning or business intelligence workloads on Databricks. Education: Bachelors Degree
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
- Dice Id: compun
- Position Id: KUMDC5779310
- Posted 3 days ago
Never miss an opportunity! Create an alert based on the job you applied for.
