Sr Data Engineer with Python in Holmdel New Jersey Hybrid

Overview

We are seeking an experienced Data Engineer to join our Data & Analytics team. This role focuses on building and optimizing scalable data pipelines and architectures using Databricks, PySpark, and Python. You will collaborate with data scientists, analysts, and business teams to deliver high-quality data solutions that enable advanced analytics and machine learning.

Key Responsibilities

• Design, develop, and maintain robust data pipelines on Databricks for ingesting, processing, and transforming large volumes of structured and unstructured data.
• Implement ETL processes using Python, PySpark, and SQL.
• Optimize and troubleshoot existing pipelines for performance, reliability, and scalability.
• Collaborate with cross-functional teams to understand data requirements and deliver solutions.
• Ensure data quality, integrity, and compliance across multiple sources.
• Integrate data from enterprise systems and external vendors using APIs and file-based ingestion.
• Work with cloud platforms (AWS/Azure) and tools like Delta Lake, Snowflake, or similar.
• Participate in code reviews, testing, and documentation to maintain high standards.
• Implement CI/CD practices for data workflows and maintain version control.