Azure Data Engineer
Milpitas, CA (Must be local to CA)
Contract
Key Responsibilities
Technical Leadership & Architecture Lead the design and implementation of Databricks Lakehouse architectures Define medallion architecture (Bronze, Silver, Gold layers) using Delta Lake Drive architectural decisions for batch and streaming data pipelines Establish coding standards, best practices, and reusable frameworks Data Engineering & Databricks Design and build scalable ETL/ELT pipelines using Databricks (PySpark/SQL/Scala) Optimize Spark jobs for performance, reliability, and cost Implement Delta Lake features (ACID, time travel, schema enforcement) Develop and manage Databricks workflows, jobs, and clusters Cloud & Platform Integration Architect Databricks solutions on Azure (preferred) or AWS Integrate Databricks with cloud storage and data services Azure: ADLS, ADF, Synapse AWS: S3, Glue, Redshift Enable BI and analytics consumption (Power BI, Tableau) Governance, Security & DevOps Implement data governance using Unity Catalog Define RBAC, data access controls, and security best practices Enable CI/CD for Databricks using GitHub / Azure DevOps Use Infrastructure-as-Code (Terraform) for environment management Leadership & Collaboration Lead, mentor, and grow data engineering teams Conduct design and code reviews Collaborate with Data Architects, Product Owners, and stakeholders Support production releases, monitoring, and incident resolution
Never miss an opportunity! Create an alert based on the job you applied for.
