PySpark Developer

Overview

On Site

$50 - $60

Contract - W2

Skills

Amazon Web Services

Apache Spark

DevOps

Big Data

SQL

Python

NoSQL

Extract

Transform

Load

PySpark

PostgreSQL

Job Details

Job Title: PySpark Developer

Location: Owings Mills, MD (Onsite)
Employment Type: Contract
Experience Level: Senior
Duration: [18+ Months]

Job Description

We are seeking a PySpark Developer with strong expertise in AWS Cloud Computing and Big Data technologies. The role requires hands-on experience with PySpark, Apache Spark, and Python to design, build, and optimize scalable data engineering solutions.

Key Responsibilities

Design, develop, and deploy PySpark-based big data solutions in cloud environments.
Architect and optimize ETL pipelines for both structured and unstructured data.
Work with SQL and NoSQL databases such as Snowflake, DB2, and PostgreSQL.
Utilize workflow schedulers like Airflow to manage complex data pipelines.
Tune Spark performance using partitioning, caching, and other optimization techniques.
Collaborate with cross-functional teams to gather requirements and deliver solutions.
Implement CI/CD pipelines, maintain version control, and ensure data governance.
Mentor junior team members and perform code reviews.

Required Skills

7+ years of experience with AWS Cloud services.
10+ years of experience in big data and distributed computing.
Advanced proficiency with PySpark, Apache Spark, and Python.
Strong hands-on experience with SQL and NoSQL databases.
Familiarity with Airflow or similar scheduling tools.
Proficiency in ETL design, data modeling, and performance tuning.
Exposure to DevOps tools, CI/CD pipelines, Docker, and Kubernetes.
Understanding of data security and compliance best practices.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.