Overview
Skills
Job Details
Title: Azure Data Engineer
Location: Remote (Texas-based candidates preferred)
Duration: Until August 31, 2026, with possible extension
Submission Deadline: October 19, 2025 (2:00 PM CST)
Job Description
We re seeking an experienced Azure Data Engineer to design, develop, and optimize enterprise-scale data solutions using the Databricks Unified Analytics Platform. The role will focus on building high-performance data pipelines, implementing robust ETL/ELT processes, and ensuring strong data governance and security across the ecosystem.
This position supports advanced analytics, data engineering, and machine learning initiatives within a data-driven organization.
Key Responsibilities
Design and develop scalable data pipelines and workflows.
Implement and maintain ETL/ELT processes for structured and unstructured data.
Optimize and tune Apache Spark jobs for performance and cost efficiency.
Integrate solutions with Azure Data Factory and other cloud services.
Automate deployments using CI/CD pipelines and version control tools.
Collaborate with cross-functional teams data scientists, analysts, and business stakeholders.
Ensure data quality, governance, lineage, and security using Delta Lake or Unity Catalog.
Contribute to data governance and metadata management initiatives.
Required Skills and Experience
Experience (Years) | Requirement |
10 | ETL/ELT workflows for structured and unstructured data |
8 | CI/CD automation and deployment |
8 | Data modeling, schema design, and database optimization |
8 | Data storage solutions: Azure Data Lake, Data Warehouse |
8 | Data validation, quality assurance, and governance |
8 | Data security and compliance (encryption, access control, auditing) |
8 | Proficiency in Python, R, and SQL |
8 | Experience with Azure Cloud, DevOps, and agile environments |
5 | Databricks (Spark, Delta Lake, Notebooks, Clusters, Jobs) |
5 | Integration with Azure Data Factory |
5 | Spark job optimization and cost-efficiency |
5 | Deep understanding of Apache Spark architecture (RDDs, DataFrames, Spark SQL) |
Preferred Qualifications
Knowledge of ML frameworks such as MLflow, Scikit-learn, or TensorFlow.
Databricks Certified Associate Developer for Apache Spark.
Microsoft Certified: Azure Data Engineer Associate.