Pyspark Developer

Overview

On Site
$40 - $50
Contract - W2

Skills

PySpark
Python
Extract
Transform
Load
Data Engineering
Apache Spark

Job Details

Our Client is seeking a highly skilled and experienced Senior PySpark Developer to join our dynamic data engineering team. This role is ideal for a motivated professional with a strong background in big data technologies, ETL processes, and database systems. The successful candidate will play a key role in designing, developing, and optimizing data pipelines and analytics solutions that support critical business operations.
Key Responsibilities:

  • Design, develop, and maintain scalable data pipelines using PySpark and Spark SQL.
  • Collaborate with data architects, analysts, and business stakeholders to understand data requirements and deliver robust solutions.
  • Optimize and troubleshoot complex ETL workflows and data transformation processes.
  • Write efficient SQL queries for data extraction, transformation, and loading.
  • Ensure data quality, integrity, and compliance with enterprise standards.
  • Participate in code reviews, testing, and deployment activities.
  • Monitor and maintain production workflows, including job scheduling and performance tuning.


Required Skills & Qualifications:

  • 5+ years of experience in data engineering or software development.
  • Strong proficiency in PySpark, Python, and Spark SQL.
  • Solid experience with SQL and relational database systems.
  • Proven expertise in ETL development and data pipeline orchestration.
  • Familiarity with job scheduling tools; Autosys experience is a plus.
  • Excellent problem-solving skills and attention to detail.
  • Strong communication and collaboration abilities.


Preferred Qualifications:

  • Experience working in the financial services industry.
  • Familiarity with cloud platforms and big data ecosystems.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.