Data Pipeline Engineer (Databricks + AWS Glue)

Remote • Posted 8 hours ago • Updated 7 hours ago
Full Time
No Travel Required
Remote
Depends on Experience
Fitment

Dice Job Match Score™

👤 Reviewing your profile...

Job Details

Skills

  • Databricks
  • PySpark
  • AWS Glue
  • Python
  • SQL
  • MongoDB
  • AWS
  • ETL
  • Data Pipelines
  • Data Ingestion
  • Data Transformation
  • Data Quality
  • Data Reconciliation

Summary

Experience: 5–10 Years
Employment Type: Full-Time

This second requirement is much more specifically centered around Databricks + AWS Glue + PySpark + Python + SQL + MongoDB, so the knockout questions should be stricter around those technologies.

About the Role

We are looking for an experienced Data Pipeline Engineer with strong hands-on expertise in Databricks, AWS Glue, PySpark, Python, SQL, and MongoDB.

The engineer will design, develop, and optimize scalable cloud-based ETL and data pipelines for processing large volumes of structured and semi-structured data.

Responsibilities

  • Design and develop scalable ETL and data pipelines using Databricks and AWS Glue.
  • Build data ingestion and transformation workflows.
  • Develop large-scale data processing solutions using PySpark, Python, and SQL.
  • Integrate data from multiple sources including MongoDB and relational databases.
  • Implement data-quality, validation, and reconciliation processes.
  • Optimize Spark and Databricks workloads for performance and scalability.
  • Build reliable cloud-native data pipelines on AWS.
  • Support production deployments, monitoring, and troubleshooting.
  • Collaborate with architects, application teams, analysts, and DevOps teams.
  • Follow Git-based development and Agile delivery practices.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
  • Dice Id: 91124577
  • Position Id: 9029133
  • Posted 8 hours ago
Contact the job poster
Lakshmi Ramachandra

Lakshmi Ramachandra

Recruiter @ Bramkas Inc.
Create job alert
Set job alertNever miss an opportunity! Create an alert based on the job you applied for.

Similar Jobs

Remote

5d ago

Easy Apply

Contract

60 - 65

Remote or Reston, Virginia

12d ago

Full-time

USD 98,614.00 - 202,850.00 per year

Remote or Boston, Massachusetts

Today

Easy Apply

Contract

$DOE

Remote

5d ago

Easy Apply

Full-time

$110,000 - $120,000

Search all similar jobs