Overview
Remote
$50+
Contract - W2
Contract - Independent
Skills
ELT
Extract
Transform
Load
Machine Learning (ML)
scikit-learn
Job Details
Azure Data Engineer
Austin, TX - Remote - NEED ONLY LOCALS
Austin, TX - Remote - NEED ONLY LOCALS
Visa: USC
CANDIDATE SKILLS AND QUALIFICATIONS
Minimum Requirements:
Years Required/Preferred Experience
10 Required Implement ETL/ELT workflows for both structured and unstructured data
8 Required Automate deployments using CI/CD tools
8 Required Collaborate with cross-functional teams including data scientists, analysts, and stakeholders
8 Required Design and maintain data models, schemas, and database structures to support analytical and operational use cases
8 Required Evaluate and implement appropriate data storage solutions, including data lakes (Azure Data Lake Storage) and data warehouses
8 Required Implement data validation and quality checks to ensure accuracy and consistency
8 Required Contribute to data governance initiatives, including metadata management, data lineage, and data cataloging
8 Required Implement data security measures, including encryption, access controls, and auditing; ensure compliance with regulations and best practices
8 Required Proficiency in Python and R programming languages
8 Required Strong SQL querying and data manipulation skills
8 Required Experience with Azure cloud platform
8 Required Experience with DevOps, CI/CD pipelines, and version control systems
8 Required Working in agile, multicultural environments
8 Required Strong troubleshooting and debugging capabilities
5 Required Design and develop scalable data pipelines using Apache Spark on Databricks
5 Required Optimize Spark jobs for performance and cost-efficiency
5 Required Integrate Databricks solutions with cloud services (Azure Data Factory)
5 Required Ensure data quality, governance, and security using Unity Catalog or Delta Lake
5 Required Deep understanding of Apache Spark architecture, RDDs, DataFrames, and Spark SQL
5 Required Hands-on experience with Databricks notebooks, clusters, jobs, and Delta Lake
1 Preferred Knowledge of ML libraries (MLflow, Scikit-learn, TensorFlow)
1 Preferred Databricks Certified Associate Developer for Apache Spark
1 Preferred Azure Data Engineer Associate
CANDIDATE SKILLS AND QUALIFICATIONS
Minimum Requirements:
Years Required/Preferred Experience
10 Required Implement ETL/ELT workflows for both structured and unstructured data
8 Required Automate deployments using CI/CD tools
8 Required Collaborate with cross-functional teams including data scientists, analysts, and stakeholders
8 Required Design and maintain data models, schemas, and database structures to support analytical and operational use cases
8 Required Evaluate and implement appropriate data storage solutions, including data lakes (Azure Data Lake Storage) and data warehouses
8 Required Implement data validation and quality checks to ensure accuracy and consistency
8 Required Contribute to data governance initiatives, including metadata management, data lineage, and data cataloging
8 Required Implement data security measures, including encryption, access controls, and auditing; ensure compliance with regulations and best practices
8 Required Proficiency in Python and R programming languages
8 Required Strong SQL querying and data manipulation skills
8 Required Experience with Azure cloud platform
8 Required Experience with DevOps, CI/CD pipelines, and version control systems
8 Required Working in agile, multicultural environments
8 Required Strong troubleshooting and debugging capabilities
5 Required Design and develop scalable data pipelines using Apache Spark on Databricks
5 Required Optimize Spark jobs for performance and cost-efficiency
5 Required Integrate Databricks solutions with cloud services (Azure Data Factory)
5 Required Ensure data quality, governance, and security using Unity Catalog or Delta Lake
5 Required Deep understanding of Apache Spark architecture, RDDs, DataFrames, and Spark SQL
5 Required Hands-on experience with Databricks notebooks, clusters, jobs, and Delta Lake
1 Preferred Knowledge of ML libraries (MLflow, Scikit-learn, TensorFlow)
1 Preferred Databricks Certified Associate Developer for Apache Spark
1 Preferred Azure Data Engineer Associate
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.