Sr Data Engineer (Databricks, Spark) – Production Support Lead

Job Title: Lead Data Engineer – Databricks | Production Support
Location: Remote (USA)
Type: 6 Months Contract + Possible extension
Rate: Upto $60 Per hour W2 + benefits

Job Overview

We are looking for a Lead Data Engineer with strong Databricks experience to own and lead production support for enterprise data platforms. This role combines hands-on data engineering with incident management, ensuring high availability, performance, and reliability of data pipelines and analytics systems.

You will act as the primary escalation point for critical issues, drive root cause analysis, and partner with engineering and business teams to maintain SLA-driven operations.

Key Responsibilities

• Lead end-to-end production support for data pipelines, ETL/ELT workflows, and Databricks environments

• Serve as the L3 escalation point for high-priority incidents and outages

• Monitor, troubleshoot, and resolve issues across batch and real-time data pipelines

• Perform root cause analysis (RCA) and implement preventive fixes

• Collaborate with Data Engineering, DevOps, and business teams for SLA adherence

• Optimize Databricks jobs, clusters, and cost efficiency

• Build and maintain monitoring, alerting, and support runbooks

• Lead on-call rotations and ensure operational readiness

• Communicate effectively with stakeholders during incidents and provide post-incident reports

Required Skills & Experience

• 8+ years of experience in Data Engineering / Production Support

• Strong hands-on experience with Databricks (Spark, Delta Lake, Workflows)

• Proficiency in Python, SQL, and Apache Spark

• Experience with cloud platforms: AWS / Azure / Google Cloud Platform

• Strong understanding of ETL/ELT pipelines and orchestration tools (Airflow preferred)

• Experience with monitoring tools, incident management, and support models (L2/L3)

• Knowledge of data quality, governance, and security best practices

Preferred Qualifications

• Experience supporting large-scale enterprise data platforms

• Familiarity with CI/CD pipelines and DevOps practices

• Exposure to real-time/streaming data processing

Soft Skills

• Strong incident leadership and decision-making under pressure

• Ability to mentor and guide junior engineers

• Excellent communication and stakeholder management skills

• Proactive mindset with focus on automation and continuous improvement

Why Join

• Opportunity to work on modern data platforms (Databricks, Spark)

• High-impact role with ownership of critical data systems

• Collaborative and fast-paced environment