Data Engineer

Title: Data Engineer

Location: Remote

Job Description:

Key Responsibilities

·       Design and implement end-to-end data architecture in Microsoft Fabric.

·       Build and operationalize Medallion Architecture (Bronze, Silver, Gold layers).

·       Develop scalable data pipelines using:

Ø  Dataflows Gen2

Ø  Pipelines (Data Factory in Fabric)

Ø  Notebooks / PySpark

·       Architect Lakehouse solutions using OneLake and Delta Tables.

·       Establish data ingestion, transformation, and serving patterns.

·       Define and enforce data governance, data quality, and security standards.

·       Optimize performance of large-scale data processing workloads.

·       Integrate data from various sources (ERP, APIs, streaming, databases).

·       Implement CI/CD pipelines and DevOps best practices for data workflows.

·       Collaborate with business teams to translate requirements into technical solutions.

·       Mentor junior engineers and guide architecture decisions.

Required Skills & Experience

·       8+ years of experience in Data Engineering and Architecture.

·       Strong hands-on experience with Microsoft Fabric or related Azure technologies:

Ø  Azure Synapse

Ø  Azure Data Factory

Ø  Azure Data Lake Storage (ADLS)

Ø  Azure Databricks

·       Deep understanding of Medallion Architecture / Lakehouse Architecture.

Technical Expertise

·       PySpark / Apache Spark

·       Advanced SQL

·       Data Modeling (Dimensional & Normalized)

Experience With

·       Delta Lake

·       Data Pipelines and Orchestration

·       Batch & Streaming Architectures

Strong Understanding Of

·       Data Governance

·       Security (RBAC, Data Masking, etc.)

·       Familiarity with Power BI integration and Semantic Models.