Data Engineer Tech Lead

Qualifications:

• 15+ years of experience (including 7+ years in Azure) in Data Engineering, ETL & Data Warehousing
• Should have excellent leadership, communication & client-facing skills
• Should have in depth knowledge on Azure Data Factory, Databricks, SSIS, Azure Functions, Logic Apps, Azure Event Hubs, PySpark, Spark Streaming, ADLS Gen2, Azure Blob, Azure Synapse, Azure SQL, SQL Server and good working knowledge on Tableau.
• Should have executed at least 2 Azure Cloud Data Warehousing projects
• Should have worked on Batch, API, Event & Streaming data processing,
• Should have working experience in building CI\CD pipelines using tools like Azure DevOps, GitHub, Jenkins, etc
• Should have working knowledge on handling various file formats like JSON, XML, CSV, Parquet, Delta, etc.
• Should have good knowledge of Agile methodology and able to work in a Scrum Master capacity.
• Should have worked in creating repeatable modules to support automation to drive dynamic data ingestion, processing, quality checks & error handling.
• Should have working knowledge on creating complex Databricks notebooks and integrate with Azure Ecosystem.
• Should have working knowledge of creating SQL queries to analyze and extract semi-structured data
• Should have in depth understanding on Data Warehousing, Data Analysis, Data Profiling, Data Quality & Data Mapping