Azure Data Tech Lead

• Architect, design, and implement scalable data platforms and pipelines on Azure and Databricks.
• Build and optimize data ingestion, transformation, and processing workflows across batch and real-time data streams.
• Work with data storage and processing technologies including data lakes, Delta Lake, and Spark using Python.
• Lead the development of complex ETL and ELT pipelines ensuring quality, reliability, and performance.
• Design and implement data models including conceptual, logical, and physical models.
• Work with relational and lakehouse systems for analytics and operational workloads.
• Define and enforce best practices for data governance, quality, security, and architecture.
• Collaborate with cross-functional teams to translate business requirements into technical solutions.
• Troubleshoot production issues and optimize platform performance.
• Mentor junior engineers and contribute to engineering standards and reusable frameworks.

Requirement/Must Have:

• 10+ years of experience in data engineering within enterprise environments.
• Strong expertise in Azure data services and platforms.
• Advanced proficiency in Apache Spark using Python.
• Strong SQL skills including query optimization and performance tuning.
• Deep understanding of ETL and ELT processes and data pipeline orchestration.
• Experience with data modeling techniques including dimensional and lakehouse models.
• Experience with both batch and real-time data processing systems.
• Strong understanding of data architecture and distributed systems.
• Ability to design end-to-end scalable data solutions.
• Excellent analytical, problem-solving, and communication skills.

Responsibilities:

• Lead data platform architecture and development.
• Ensure high performance and reliability of data pipelines.
• Collaborate with stakeholders across business and technical teams.
• Guide and mentor engineering teams.
• Establish and maintain best practices in data engineering.

Should Have:

• Experience with CI/CD tools and version control systems.
• Familiarity with infrastructure as code tools.
• Experience with data governance and cataloging tools.
• Exposure to machine learning or business intelligence workloads.

Skills:

• Strong programming skills in Python and SQL.
• Expertise in big data processing frameworks.
• Data modeling and architecture design skills.
• Performance tuning and optimization.
• Strong communication and leadership abilities.

Qualification And Education:

• Bachelor s degree in Computer Science, Engineering, or related field.
• Master s degree preferred.