Sr. Databricks Engineer

Overview

Hybrid
Depends on Experience
Accepts corp to corp applications
Contract - Independent
Contract - W2
Able to Provide Sponsorship

Skills

Databricks
PySpark
Cloud

Job Details

Title: Databricks Engineer Location: Los Angeles, CA Duration: 1 year +

Need only locals

Key Responsibilities Data Pipeline Development: Design and implement robust ETL/ELT pipelines using Databricks, PySpark, and Delta Lake to process structured and unstructured data efficiently.
Performance Optimization: Tune and optimize Databricks clusters and notebooks for performance, scalability, and cost-efficiency.
Collaboration: Work closely with data scientists, analysts, and business stakeholders to understand data requirements and deliver solutions that meet business needs.
Cloud Integration: Leverage cloud platforms (AWS, Azure, Google Cloud Platform) to build and deploy data solutions, ensuring seamless integration with existing infrastructure.
Data Modeling: Develop and maintain data models that support analytics and machine learning workflows.
Automation & Monitoring: Implement automated testing, monitoring, and alerting mechanisms to ensure data pipeline reliability and data quality.
Documentation & Best Practices: Maintain comprehensive documentation of data workflows and adhere to best practices in coding, version control, and data governance.

Required Qualifications Experience: 5+ years in data engineering, with hands-on experience using Databricks and Apache Spark.
Programming Skills: Proficiency in Python and SQL; experience with Scala is a plus.
Cloud Platforms: Strong experience with cloud services such as AWS (e.g., S3, Glue, Redshift), Azure (e.g., Data Factory, Synapse), or Google Cloud Platform.
Data Engineering Tools: Familiarity with tools like Airflow, Kafka, and dbt.
Data Modeling: Experience in designing data models for analytics and machine learning applications.
Collaboration: Proven ability to work in cross-functional teams and communicate effectively with non-technical stakeholders.
Primary Skill Set:
Databricks, Apache Spark, Python, SQL, Scala (optional), ETL/ELT development, Delta Lake, Cloud platforms (AWS, Azure, Google Cloud Platform), Data modeling, Cross-functional collaboration, Communication

Secondary Skill Set:
Airflow, dbt, Kafka, Hadoop, MLflow, Unity Catalog, Delta Live Tables, Cluster optimization, Data governance, Security and compliance, Databricks certifications

Thanks & Regards,

Naresh

Global IT Family
Email:

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.