Overview
Skills
Job Details
Greetings from Caritatech
Title: Data Engineer / Pyspark Developer
Duration: 12+ months W2 contract
Visa Status: Any
-
Expertise in PySpark with a strong background in building scalable data pipelines
-
Solid hands-on experience with Spark Streaming for real-time data ingestion and processing
-
Practical knowledge of AWS services including EMR, EKS, and Airflow for orchestrating and managing data workflows
-
Proficient in using Apache NiFi for efficient data ingestion, routing, and transformation
-
Hands-on experience with Iceberg tables and managing S3-based data lake architectures
-
Familiar with AWS Aurora PostgreSQL and capable of integrating with external systems via APIs
-
Strong command over Python, SQL, and T-SQL for data manipulation and querying
-
Proven track record in optimizing and enhancing performance of distributed data jobs
-
Previous experience working in client-facing or consulting roles, with an ability to understand and address customer needs
-
Excellent interpersonal and communication skills, with the ability to explain complex technical concepts clearly
-
A self-motivated individual who is enthusiastic about innovation and continuous improvement in data engineering practices
-
Strong analytical mindset with a keen eye for problem-solving and detail orientation