Azure Data Factory Developer

Introduction:

As an Azure Data Factory Developer, you will be responsible for designing, developing, and maintaining data pipelines and data architecture to support Marketing organizations. You will work closely with business stakeholders to understand their data needs, and then translate those requirements into technical solutions using Azure Data Factory, Azure Data Lake, and other related technologies.

Responsibilities:

• Design and implement data models for data logging, validation, and transformations
• Build and maintain data pipelines for data ingestion, processing, and consumption
• Define data quality and master data rules for data matching, merging, and de-duplication
• Collaborate with business stakeholders to understand complex business systems and design data architecture
• Communicate technical concepts to non-technical audiences and translate business requirements into technical specifications
• Implement data warehousing solutions using Azure Data Factory, Azure Data Lake, and Azure Databricks
• Write PySpark/Python code to process and transform data from various sources
• Utilize Azure Databricks platform to build data analytics solutions for performance and scale
• Manage source code control using Git branching and CI/CD deployments using DevOps Pipelines
• Create technical documentation including solutions design, architecture diagrams, and code comments
• Guide and mentor other developers in the team and resolve technical issues in a timely manner

Requirements:

Required Skills:

• 10 to 12 years of experience in data platforms, data architecture, and analytics
• Strong expertise in ADF, Business Intelligence, Data Modeling, SQL Azure, Python, PySpark
• Problem-solving skills and ability to communicate complex data problems in simple solutions
• Experience with DevOps, Data Quality, Data Lake, Data Engineering, and Big Data
• Proficiency in Git, Specification Gathering, Data Analysis, and Communication
• Knowledge of Data Warehouse, Data Wrangling, Databricks, Lambda Architecture, and SQL
• Experience in scripting languages and modern data warehouse/Lambda architecture concepts

Preferred Skills:

• Understanding of Azure Data Lake Analytics & U-SQL, Azure SQL DW
• Ability to work with Azure Databricks platform for data analytics solutions
• Experience with PowerShell or similar scripting languages
• Familiarity with source code control using Git and CI/CD deployments