Azure Data Engineer (Databrick expertise W2 only)

Azure Data Engineer

Remote/Nearshore - EST hours

# of positions: 05

Key Responsibilities:

• Design, implement, and manage scalable data solutions on Microsoft Azure, adhering to enterprise standards.
• Develop, analyze, and optimize Power Query scripts for efficient ETL (Extract, Transform, Load) processes.
• Architect and implement Azure SQL databases ensuring data integrity, security, and performance.
• Execute advanced SQL tuning and query optimization techniques to improve performance across large datasets.
• Build and manage data pipelines using Azure Data Factory, ensuring seamless data orchestration, transformation, and lineage tracking.
• Design and maintain Azure Analysis Services models to enable robust analytics and reporting capabilities.
• Utilize Azure Databricks (PySpark) for large-scale data processing and analytics workloads.
• Develop and maintain Databricks Asset Bundles and YAML-based configuration-driven architectures for CI/CD integration, promoting code modularity and automation.
• Apply performance tuning and cost optimization techniques in Databricks, including cluster sizing, Delta caching, job cluster vs interactive cluster usage, and partitioning strategies.
• Implement data governance and access control using Unity Catalog, ensuring regulatory compliance and security enforcement.
• Build and optimize Power BI dashboards and reports using DAX for business insights and visual storytelling.
• Conduct unit testing, validation, and documentation of all data solutions to ensure quality and reproducibility.

Qualifications

• Bachelor s or master s degree in computer science, IT, or a related discipline.
• 5+ years of experience in data engineering with at least 3 years in Azure Data stack.
• Strong expertise in Azure cloud services, including Azure SQL, ADF, AAS, and Azure Storage.
• Hands-on experience with Databricks, including PySpark, cluster management, and CI/CD practices using Asset Bundles.
• Proven experience in SQL performance tuning, indexing strategies, and workload analysis.
• Proficient in Power BI, including advanced DAX and data modeling for self-service BI.
• Good understanding of data warehousing principles, dimensional modeling, and implementation best practices.
• Excellent communication skills and a collaborative mindset to work effectively in cross-functional teams.

Preferred Skills

• Microsoft Azure Data Engineer Certification or equivalent.
• Knowledge of data lake architecture and implementation on Azure.
• Familiarity with data governance, PII handling, and RBAC enforcement.
• Exposure to DevOps, CI/CD pipelines, Git version control, and release management using YAML workflows.