Data Engineer with Azure Data Factory workloads (In person interview)

Overview

Remote
Hybrid
$50 - $60
Contract - W2
Contract - Independent

Skills

Azure Data Factor

Job Details

Scope of Work:

Client is looking to ingest datasets into a scalable and efficient data lake on the A WS platform and Azure platforms to ingest and process a variety of data sources, including ERP data, labor/workforce data, business data, affordability data, health data, and more. The data lake will follow a medallion architecture, with these resources focused on hydrating the bronze and silver layers. The selected data engineering staff will be responsible for designing and implementing the data ingestion workflows, data quality checks, and data transformation pipelines to populate the bronze and silver layers of the data lake. Key requirements include expertise in AWS data services (e.g., S3, Glue, Athena, Redshift), Azure Data Factory, data modeling and optimization, data management, data security, data governance, all in close collaboration with technologists, data analysts, and business stakeholders. The successful vendor will demonstrate a proven track record in building and maintaining large-scale, enterprise-grade data lake solutions.

Responsibilities:

  • Design and implement data ingestion workflows from various sources.
  • Build robust, scalable, and efficient ELT processes using AWS services.
  • Design data models and manage schemas for the bronze and silver layers of the medallion architecture.
  • Implement data quality checks, data lineage tracking, and metadata management.
  • Automate data pipeline workflows using tools like AWS Glue, AWS Step Functions, or Apache Airflow.
  • Collaborate with technical staff, data analysts, data scientists, and business stakeholders.
  • Support existing Azure Data Factory workloads.
  • Maintain comprehensive documentation of data lake architecture, data models, and pipeline workflows.
  • Apply domain expertise in government, business, workforce, or health data (preferred).
  • Leverage 3 5 years of experience in building and maintaining large-scale, enterprise-grade data lake solutions on AWS.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.