Overview
Remote
$60 - $70
Contract - W2
Contract - Independent
Contract - 12 Month(s)
No Travel Required
Skills
Data Scientists
AWS Glue
PySpark
DynamoDB
Snowflake
Python
Lambda
Job Details
Job Description :
- Collaborate with cross-functional teams including Data Scientists, Analysts, and Engineers to gather data requirements and build scalable data solutions.
- Design, develop, and maintain complex ETL pipelines using AWS Glue and PySpark, ensuring efficient data processing across batch and streaming workloads.
- Integrate and manage data storage and retrieval using AWS DynamoDB and Snowflake, optimizing for performance and scalability.
- Ensure data integrity, quality, and security across data pipelines, applying best practices for encryption, IAM, and compliance.
- Monitor and troubleshoot pipeline issues, continuously optimizing for cost and performance across AWS services.
- Stay current with advancements in AWS Glue, PySpark, and data infrastructure tools, and recommend improvements where applicable.
Experience / Minimum Requirements:
- 8+ years of experience as a Data Engineer, with strong hands-on expertise in AWS Glue, PySpark, AWS DynamoDB, and Snowflake.
- Deep understanding of Spark architecture, distributed processing, and performance tuning techniques.
- Strong grasp of data modeling, schema design, and data warehouse concepts.
- Experience with AWS data ecosystem including S3, Lambda and Glue Catalog.
- Proficiency in Python (PySpark) for data transformation and automation tasks.
- Familiarity with CI/CD practices and infrastructure-as-code tools such as Terraform is a plus.
- Excellent communication and problem-solving skills, with the ability to work independently and in a team environment.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.