Data Engineer with Azure and Databricks

One of our oprestigious clients in NYC, is looking for an AZURE DATA ENGINEER

Position: Databricks  Engineer

NYC HYBRID 

Responsibilities:

• Develop, optimize, and maintain  Databricks notebooks for ETL and data transformation.
• Design and implement scalable data pipelines using Databricks and Azure Data Factory.
• Apply strong knowledge of distributed computing concepts to large-scale data processing.
• Work with data lake architectures and Delta Lake for reliable and performant data storage.
• Automate data quality checks, monitoring, and logging.
• Integrate data from multiple cloud sources such as Azure Blob, Data Lake Storage, and SQL/NoSQL databases.
• Perform data modeling and schema design for analytical workloads.
• Implement CI/CD pipelines for Databricks notebooks and jobs.
• Manage workspace administration, cluster management, and job orchestration in Databricks.

Qualifications:

• Proven experience with Databricks for large-scale data processing and analytics..
• Familiarity with cloud data platforms, preferably Azure.
• Understanding of data engineering best practices and distributed systems.
• Experience with data pipeline orchestration and automation.