DataBricks Engineer

Hybrid in Jersey City, NJ, US • Posted 1 day ago • Updated 1 day ago
Contract W2
12 Months
No Travel Required
Hybrid
$60 - $65/hr
Fitment

Dice Job Match Score™

🔢 Crunching numbers...

Job Details

Skills

  • Databricks
  • Apache Kafka
  • Python
  • PySpark
  • SQL
  • Data Warehouse
  • Data Modeling
  • Data Governance
  • DevOps
  • Data Engineering
  • Continuous Integration
  • Continuous Delivery

Summary

Job Description: Databricks Engineer – Lead / Senior Lead

Job Title: Databricks Lead / Senior Lead Engineer
Experience: 8–12+ years (flexible based on level)
Job Type: Contract
Work Location: Lawrenseville, NJ

Job Summary

We are seeking an experienced Databricks Engineer for Lead and Senior Lead positions to design, develop, and optimize scalable data engineering solutions using Databricks, Apache Spark, Python, and SQL. The ideal candidate will have strong hands-on expertise in cloud-based data platforms, data pipeline development, data modeling, and performance optimization, along with the ability to lead technical teams and collaborate with business stakeholders.

Key Responsibilities

  • Design, develop, and maintain scalable data pipelines using Databricks, PySpark, Python, and SQL.

  • Build and optimize ETL/ELT workflows for large-scale data processing.

  • Develop and manage Delta Lake tables, Lakehouse architectures, and data transformation frameworks.

  • Implement data ingestion, integration, cleansing, validation, and quality checks.

  • Optimize Spark jobs, cluster configurations, and query performance.

  • Work with cloud platforms such as AWS, Azure, or Google Cloud.

  • Implement workflow orchestration, monitoring, logging, and error-handling mechanisms.

  • Ensure data security, governance, access control, and compliance with organizational standards.

  • Collaborate with data architects, analysts, product teams, and business stakeholders.

  • Review code, establish engineering best practices, and mentor junior engineers.

Required Skills

  • Strong hands-on experience with Databricks and Apache Spark.

  • Advanced proficiency in PySpark, Python, and SQL.

  • Experience with Delta Lake, Lakehouse architecture, and ETL/ELT pipelines.

  • Strong understanding of data warehousing, data modeling, and distributed data processing.

  • Experience with cloud-based data platforms, preferably Azure Databricks or AWS Databricks.

  • Knowledge of CI/CD, Git, automated testing, and deployment practices.

  • Experience with data orchestration tools and workflow scheduling.

  • Strong analytical, problem-solving, and communication skills.

Role-Specific Expectations

1. Databricks Lead Engineer

  • Lead day-to-day technical development and delivery of data engineering solutions.

  • Guide a team of data engineers and conduct code reviews.

  • Translate technical requirements into scalable implementation plans.

  • Troubleshoot complex pipeline and performance issues.

  • Coordinate with cross-functional teams to ensure timely project delivery.

  • Maintain coding standards, documentation, and engineering best practices.

2. Databricks Senior Lead Engineer

  • Own the technical direction and architecture of complex Databricks solutions.

  • Lead multiple workstreams and guide teams through design and implementation.

  • Define engineering standards, reusable frameworks, and performance optimization strategies.

  • Drive cloud data platform modernization and large-scale data transformation initiatives.

  • Collaborate with architects and senior stakeholders on technical roadmaps and solution design.

  • Manage technical risks, delivery dependencies, and escalations.

  • Mentor technical leads and senior engineers and ensure high-quality delivery.

Preferred Qualifications

  • Databricks certifications or relevant cloud certifications.

  • Experience with Azure Data Factory, AWS Glue, or similar orchestration and integration tools.

  • Knowledge of streaming technologies such as Kafka or Structured Streaming.

  • Experience with Unity Catalog, data governance, and access management.

  • Familiarity with Terraform, DevOps, and infrastructure automation.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
  • Dice Id: 91173647
  • Position Id: 9143506
  • Posted 1 day ago

Company Info

About GState Consulting LLC

At GState Consulting, we are a strategic technology and business consulting firm committed to accelerating innovation and enabling long-term growth. We design and implement intelligent, purpose-driven solutions that transform how organizations operate and compete.

By combining industry insight with a results-oriented mindset, we help our clients lead confidently in an ever-evolving digital landscape.

Our Vision
To be the trusted partner that empowers organizations to innovate, adapt, and lead with confidence in a connected, technology-driven world.

Contact the job poster
AK

Abhishek Kailash Chandra

Recruiter @ GState Consulting LLC

Create job alert
Never miss an opportunity! Create an alert based on the job you applied for.