Overview
Remote
Depends on Experience
Contract - Independent
Contract - W2
Skills
Apache Spark
Databricks
Delta Lake
Declarative Pipelines
Autoloader
PySpark
SQL
Spark optimization
Job Details
Key Responsibilities
- Lead implementation of modern data solutions on Databrick
- Implement data governance, access control, and lineage using Unity Catalog.
- Optimize ETL/ELT processes and Spark workloads for high performance.
- Collaborate with data engineers, analysts, and scientists to ensure business and technical alignment.
- Provide guidance on cloud data architecture and CI/CD best practices.
Required Skills & Experience
- 8 10 years in Big Data Engineering with Apache Spark; 3 4 years hands-on with Databricks.
- Expertise in Delta Lake, declarative pipelines, batch/streaming data integration and Autoloader.
- Proficient in PySpark, SQL, Spark optimization, and data modeling.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.