Data Engineer - Pyspark - REMOTE - W2

Overview

Contract - Independent
Contract - W2
Contract - 12 month(s)

Skills

AWS
spark
airflow
pyspark
cicd
EKS
EMR

Job Details

Hi,
Greetings from Caritatech

Title: Data Engineer / Pyspark Developer
Duration: 12+ months W2 contract
Visa Status: Any

  • Expertise in PySpark with a strong background in building scalable data pipelines

  • Solid hands-on experience with Spark Streaming for real-time data ingestion and processing

  • Practical knowledge of AWS services including EMR, EKS, and Airflow for orchestrating and managing data workflows

  • Proficient in using Apache NiFi for efficient data ingestion, routing, and transformation

  • Hands-on experience with Iceberg tables and managing S3-based data lake architectures

  • Familiar with AWS Aurora PostgreSQL and capable of integrating with external systems via APIs

  • Strong command over Python, SQL, and T-SQL for data manipulation and querying

  • Proven track record in optimizing and enhancing performance of distributed data jobs

  • Previous experience working in client-facing or consulting roles, with an ability to understand and address customer needs

  • Excellent interpersonal and communication skills, with the ability to explain complex technical concepts clearly

  • A self-motivated individual who is enthusiastic about innovation and continuous improvement in data engineering practices

  • Strong analytical mindset with a keen eye for problem-solving and detail orientation


Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.