Senior Data Engineer (PySpark / Databricks / AWS Glue)

Job Title: Senior Data Engineer (PySpark / Databricks / AWS Glue)

Location: Onsite New York, NY

Locals are highly preferable

Experience: Minimum 10 12 years

Job Description:

We are seeking a highly experienced Senior Data Engineer to support our Federal client. The ideal candidate will have deep expertise in PySpark, Databricks, and AWS Glue, with a strong background in designing and implementing scalable data pipelines and analytics solutions in cloud environments. This is an onsite role based in New York, and only local candidates will be considered.

Key Responsibilities:

Design, develop, and maintain large-scale data pipelines using PySpark, Databricks, and AWS Glue.

Collaborate with data architects, analysts, and stakeholders to define data models and ETL processes.

Optimize and troubleshoot data workflows for performance, scalability, and reliability.

Implement best practices for data governance, security, and quality.

Work closely with cross-functional teams to integrate data from multiple sources.

Support production environments and ensure data integrity and accuracy.

Required Skills & Qualifications:

10 12+ years of overall experience in Data Engineering or related roles.

Strong hands-on experience with PySpark, Databricks, and AWS Glue.

Proficiency with AWS services (S3, Lambda, Redshift, EMR, etc.).

Solid understanding of ETL processes, data warehousing, and data modeling.

Experience working with large-scale distributed systems and big data processing frameworks.

Strong SQL skills and experience with performance tuning.

Excellent communication and collaboration skills.

Must be able to work onsite in New York, NY.