Overview
On Site
Depends on Experience
Accepts corp to corp applications
Contract - W2
Contract - 12 Month(s)
Skills
Amazon S3
Amazon Redshift
Databricks
Python
Scala
Extract
Transform
Load
Machine Learning (ML)
Apache Spark
Data Lake
Delta Lake
Job Details
Senior ETL Databricks Developer
Long Term Contract
Foster City, CA - ONSITE
We need someone local to Bay area CA
We are seeking a Senior ETL Databricks Developer to design and optimize high-performance data pipelines using Databricks on AWS. The role demands expertise in building scalable data solutions, leveraging modern data lake architectures, and integrating with advanced tools to support enterprise data workflows.
Key Responsibilities
- Design, develop, and optimize ETL data pipelines using Databricks and AWS.
- Implement and manage data lake solutions with Delta Lake for efficient storage and processing.
- Utilize Spark (Python/Scala) for large-scale data processing and performance tuning.
- Develop robust data models to support business requirements and analytics.
- Integrate CI/CD pipelines for automated data workflow deployments.
- Collaborate with cross-functional teams to integrate MLflow and other tools for enhanced data operations.
Required Skills and Experience
- Extensive experience with Databricks, Spark, and Delta Lake on AWS.
- Proficiency in Python and/or Scala for data pipeline development.
- Strong knowledge of modern data lake architectures and ETL frameworks.
- Expertise in data modeling and performance optimization for large-scale datasets.
- Hands-on experience with CI/CD for data workflows.
- Familiarity with MLflow for managing machine learning lifecycles.
Preferred Qualifications
- Experience with AWS cloud services (e.g., S3, Redshift, Glue).
- Knowledge of data governance and security best practices.
- Ability to work in a fast-paced, collaborative onsite environment.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.