Senior Databricks Data Engineer (AWS)

Remote • Posted 1 hour ago • Updated 1 hour ago
Full Time
No Travel Required
Remote
Depends on Experience
Fitment

Dice Job Match Score™

🔢 Crunching numbers...

Job Details

Skills

  • ETL
  • Databricks
  • pyspark
  • spark
  • AWS

Summary

Job Title: Senior Databricks Data Engineer (AWS)

Location Remote

Job type: Fulltime

 

Job Description

Must Have Technical/Functional Skills

  • 8+ years of experience in Data Engineering, ETL/ELT, and Data Warehousing.
  • 5+ years of hands-on experience with Databricks, PySpark, and Apache Spark.
  • Strong expertise in Delta Lake, Lakehouse Architecture, and large-scale data processing.
  • Advanced SQL and PL/SQL development skills.
  • Experience designing and implementing scalable batch and real-time data pipelines.
  • Strong knowledge of dimensional, relational, and analytical data modeling.
  • Experience with Spark SQL, DataFrames, Hive, and distributed computing frameworks.
  • Hands-on experience with AWS services, including S3, IAM, Glue, Lambda, and Redshift.
  • Strong understanding of Databricks-AWS integrations and cloud-native architectures.
  • Experience with data quality, performance tuning, optimization, and troubleshooting.
  • Knowledge of CI/CD, Git, DevOps practices, and deployment automation.
  • Excellent analytical, problem-solving, communication, and stakeholder management skills.
  • Prior experience leading technical teams and mentoring junior engineers.

Roles & Responsibilities

  • Gather and analyze business and technical requirements from stakeholders and clients.
  • Design end-to-end data solutions, architectures, and scalable data models.
  • Develop and optimize data ingestion, transformation, and analytics pipelines using Databricks and PySpark.
  • Create and maintain conceptual, logical, and physical data models.
  • Lead solution design activities, including High-Level and Low-Level Design documentation.
  • Implement data quality, monitoring, governance, and performance optimization best practices.
  • Collaborate with Product Owners, Architects, and Business Analysts to define project scope and delivery plans.
  • Provide technical leadership, mentoring, and guidance to onshore and offshore teams.
  • Conduct design reviews, code reviews, and troubleshooting sessions to ensure solution quality.
  • Drive continuous improvements in platform performance, scalability, reliability, and cost optimization.
  • Support production deployments, issue resolution, and root cause analysis activities.
  • Contribute to technology roadmaps and recommend innovative solutions aligned with business goals.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
  • Dice Id: 91137892
  • Position Id: 9142553
  • Posted 1 hour ago
Contact the job poster
RS

Ravi Shirodkar

Recruiter @ iPeople Infosystems LLC

Create job alert
Never miss an opportunity! Create an alert based on the job you applied for.