Azure data engineer

Job description

Job Title Azure Data Engineer

Desired Skills

1Possess a minimum of 5 years of experience in data engineering and development

2Strong experience in designing and implementing data solutions using Azure data services Azure Databricks Azure Data Lake Azure Synapse Analytics Azure Data Factory

3Have a strong background in Retail CPG domain

4Proficiency in SQL TSQL and data modelling techniques

5Experience with PySpark and orchestration tools like Apache Airflow are highly desirable

6Hands on experience with scripting languages such as Python for data processing and manipulation

7Strong analytical and problemsolving skills

8Excellent communication and interpersonal skills with ability to work effectively in a collaborative team environment

9Knowledge on Databricks Unity Catalog and visualization tools like Power BI is a plus

Roles and responsibilities

1Develop and optimize data workflows using PySpark ADLS and Databricks workflows to support business objectives

2Implement and maintain data pipelines to ensure efficient data processing and storage

3Collaborate with crossfunctional teams to gather requirements and deliver data solutions that meet business needs

4Provide technical expertise in the design and implementation of data solutions ensuring scalability and performance

5Monitor and troubleshoot data workflows to ensure reliability and accuracy of data processing

6Apply Spark in Python for advanced data processing and analytics

7Build automation solutions and utilities to eliminate manual efforts and repetitive tasks

8Utilize Databricks SQL for effective data management and querying

9Implement monitoring solutions in Azure to track resource utilization performance and health of data workflows

10Manage and organize data assets using Databricks Unity Catalog

11Leverage Azure DevOps for continuous integration and deployment of data solutions

12Secure data workflows using Azure Key Vault and ensure compliance with data security standards