Data Engineer (Apache Spark, Scala, Python, AWS)

Overview

On Site
$63 - $68
Contract - W2
Contract - 12 Month(s)
No Travel Required

Skills

Data Engineer
Building Data Pipelines
Scalable ETL Solutions
Apache Spark
Scala
Python
AWS
ETL
S3
EMR
Glue
Lambda
CI/CD
Docker
Kubertness

Job Details

Job Description:
Client is seeking a skilled Data Engineer to join a high-performing data team for a leading financial services client. The ideal candidate will have strong expertise in building data pipelines and scalable ETL solutions using Apache Spark, Scala, Python, and AWS technologies.

Required Skills:
Strong experience with Apache Spark, Scala, and Python
Hands-on expertise in AWS services related to data engineering (e.g., S3, EMR, Glue, Lambda)
Solid understanding of data modeling, ETL development, and pipeline orchestration
Proven experience with large-scale data processing and performance tuning

Nice to Have:
Familiarity with CI/CD tools like Git and Jenkins
Experience with containerization tools such as Docker and Kubernetes
Exposure to real-time processing tools like Kafka and Flink

Responsibilities:
Design, develop, and optimize scalable data pipelines using Apache Spark and Scala
Build high-performance data applications and implement ETL workflows in the AWS Cloud environment
Develop reusable Python scripts and libraries to support data transformation and analysis
Collaborate closely with data scientists, analysts, and business stakeholders to understand requirements and deliver quality solutions
Ensure data accuracy, integrity, security, and compliance with enterprise data governance policies
Automate batch and real-time data ingestion workflows
Contribute to the development of real-time streaming data solutions where applicable

We are an equal opportunity employer. It is our policy to provide employment, compensation, and other benefits related to employment without regard to race, color, religion, sex, gender, national or ethnic origin, disability, veteran status, age, genetic information, citizenship, or any other basis prohibited by applicable federal, state, or local law.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.