Position: Data Lead Engineer
Location: Bethlehem ,NJ (Onsite)
Databricks / PySpark / AWS / ETL
Lead Data Engineer with strong expertise in Databricks, PySpark, Delta Lake, and SQL to design, develop, and optimize scalable data pipelines and ETL/ELT solution
The ideal candidate will have experience collaborating with business stakeholders and coordinating onsite and offshore delivery teams. Informatica Intelligent Data Management Cloud (IDMC) experience is preferred to support legacy integration and migration initiatives.
Key Responsibilities
• Design and develop scalable data pipelines using Databricks, PySpark, and Delta Lake.
• Build and optimize ETL/ELT workflows on cloud data platforms.
• Perform data analysis, transformation, and integration across multiple systems.
• Collaborate with business stakeholders and architecture teams to translate requirements into technical solutions.
• Lead ETL design, optimization, and migration initiatives, including Informatica-to-Databricks migrations.
• Perform data quality checks, reconciliation, and performance tuning.
• Support deployment, monitoring, and production operations.
• Investigate production issues and coordinate incident resolution.
• Coordinate requirements, delivery, and technical activities across onsite and offshore teams.
Required Qualifications
• 10+ years of experience in data engineering and ETL development.
• 8+ years of experience coordinating with onsite and offshore teams and business stakeholders on requirements and delivery.
• Strong hands-on expertise in Databricks, including PySpark, notebooks, and Delta tables.
• Advanced SQL skills, including complex querying, query optimization, and analytics.
• Strong understanding of ETL/ELT design and data warehousing concepts.
• Experience in data modeling and pipeline design.
• Working knowledge of AWS cloud platforms.
• Experience with Unix shell scripting.
• Experience working in an onshore/offshore delivery model.
• Strong communication, collaboration, and problem-solving skills.
