Data Bricks Engineer

Location : Cary ,nc 

Introduction:

The Data Bricks Engineer will be responsible for designing, developing, and deploying scalable ETL/ELT pipelines using Databricks, Apache Spark, and PySpark. They will collaborate with various teams to build and optimize data transformation solutions and ensure high-quality data operations on Azure-based cloud platforms.

Responsibilities:

• Design, develop, and deploy scalable ETL/ELT pipelines using Databricks, Spark, and PySpark.
• Build and optimize SQL-based data transformation and analytics solutions.
• Collaborate with product owners, data architects, analysts, and business teams on data models and workflows.
• Develop Delta Lake structures and enterprise data pipelines.
• Implement job orchestration, monitoring, and data quality controls.
• Maintain CI/CD pipelines for Databricks notebooks, jobs, and workflows.
• Support Azure-based cloud data platforms and ensure reliable, high-performing data operations.

Requirements:

Required Skills:

• Databricks and Apache Spark
• SQL and data analysis
• ETL/ELT development and optimization
• Data modeling and performance tuning
• Python or Scala
• Azure cloud-based data platforms
• Production support and operational monitoring

Preferred Skills:

• Extract, Transform, Load (ETL)
• Continuous Integration
• Data Quality
• Orchestration
• Continuous Delivery
• Data Analysis
• Analytics
• Cloud Computing
• Optimization
• Workflow
• Performance Tuning
• Microsoft Azure
• PySpark