Enterprise Data Architect

The client is seeking a strong Data Architect to be part of their data product development team. It consists of data engineers who can assist in translating business requirements into data products. Create efficient, reusable data pipelines; work with product owners and stakeholders to design, develop, and implement data supply chains in enterprise data warehouse environments using Microsoft Azure, ADF, Python, Pyspark, Spark SQL and Databricks.

Responsibilities::

• Ensure data quality, reliability, orchestration of data pipelines, CI/CD implementation, and performance/optimization of data pipelines for efficiency, while collaborating with cross-functional teams to integrate data sources and optimize pipelines.  
• Also,  this engineer must provide technical expertise and support for Azure Databricks-related projects.
• Follow technical best practices and coding standards, job orchestration & monitoring, and produce mandatory artifacts.

Requirements:

• Bachelor's degree in computer science, information technology, or a related field.
• Prior experience with data lakes and data warehouses, working knowledge of dimensional modeling is a must.
• Demonstrate an advanced understanding of data engineering practices, including ETL/ELT, data integrations, reusable data pipelines, and data management methods.
• Advanced skills required in Pyspark, Python, Spark SQL, Dimensional models, optimization in Azure, and Databricks.
• Ability to service various business stakeholders, internal and external partners to support data demands.
• Healthcare experience with a data-centric motive is a plus.
• 8+ years of experience in data engineering with the Azure Databricks ecosystem.
• Strong knowledge of Azure data services, including ADF, Databricks, Delta Lake, delta live tables, Power BI, and Azure SQL Server
• Experience with data processing languages such as SQL, Python, Pyspark and Spark SQL
• Experience with agile methodologies on Azure DevOps and continuous integration/continuous delivery (CI/CD)
• Strong problem-solving and analytical skills; excellent communication and collaboration skills
• Self-motivated and a self-starter with strong ability to multitask projects/tasks effectively.
• Ability to work independently and collaborate effectively in a team environment.