Data Engineer (Databricks)

Boston, MA, US • Posted 10 hours ago • Updated 10 hours ago
Full Time
No Travel Required
On-site
Depends on Experience
Fitment

Dice Job Match Score™

⭐ Evaluating experience...

Job Details

Skills

  • Amazon Web Services
  • Databricks
  • Kafka
  • pyspark
  • data lake
  • medallion architecture

Summary

Role: Data Engineer (Databricks)
Location: Boston, MA (Day One Onsite)
Fulltime
 
Key Responsibilities

  • Design, develop, and optimize ETL/ELT pipelines in Databricks.
  • Implement real-time and batch data processing solutions using Apache Spark and Delta Lake.
  • Develop PySpark/Scala-based data transformation scripts for large-scale data processing.
  • Ensure data quality, performance tuning, and cost optimization within Databricks.
  • Implement CI/CD pipelines for Databricks workflows using Terraform, GitHub Actions, or Azure DevOps.
  • Monitor, troubleshoot, and optimize Databricks clusters, jobs, and queries.
  • Collaborate with Data Architects, Business Analysts, and DevOps teams to align solutions with business needs.

 
Required Skills

  • Hands on experience in Databricks (AWS) – Delta Lake, Medallion architecture, Coding skills etc.
  • Proficiency in PySpark for data engineering workflows.
  • Hands-on experience with file handling, Kafka, APIs, and streaming data processing.
  • Proficiency in SQL for querying and performance tuning.
  • Experience in DevOps and CI/CD pipelines for Databricks.
  • Understanding of Data Governance, Security, and Access Control in Databricks.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
  • Dice Id: 91088983
  • Position Id: XFAR-313
  • Posted 10 hours ago
Create job alert
Set job alertNever miss an opportunity! Create an alert based on the job you applied for.

Similar Jobs

Remote

Today

Easy Apply

Full-time

$50 - $60

Hartford, Connecticut

22d ago

Full-time

USD 109,300.00 - 180,200.00 per year

Hybrid in New York, New York

Today

Easy Apply

Full-time

$80,000 - $100,000

Bedminster, New Jersey

Today

Easy Apply

Full-time

Search all similar jobs