Senior AWS PySpark Developer

Hybrid in South San Francisco, CA, US • Posted 9 hours ago • Updated 9 hours ago

Contract W2

Travel Required

Hybrid

$70/hr

Fitment

Dice Job Match Score™

📊 Calculating match score...

Job Details

Skills

AWS Cloud Platform
Data Engineer
Data Developer
Python
Big Data Tools: AWS Athena
AWS Redshift
PostgreSQL
AWS Glue
AWS Data Pipeline
AWS Lake Formation
Kubernetes
Docker
Amazon ECR
Amazon ECS
Amazon EKS
Tableau
PowerBI
Kinesis
Spark-steaming
Amazon Kinesis
Amazon Redshift
Amazon Web Services
Machine Learning (ML)
Microsoft Power BI
TensorFlow
Streaming

Summary

Job Description:

We are seeking an experienced Sr. AWS PySpark Developer with 8 -10 years of experience to design, build, and optimize our data pipelines and analytics architecture. The ideal candidate will have a strong background in data wrangling and analysis, with a deep understanding of AWS data services.

Key Responsibilities:

Design, build, and optimize robust data pipelines and data architecture on the AWS cloud platform.
Wrangle, explore, and analyze large datasets to identify trends, answer business questions, and pinpoint areas for improvement.
Develop and maintain a next-generation analytics environment that provides a self-service, centralized platform for all data-centric activities.
Formulate and implement distributed algorithms for effective data processing and trend identification.
Configure and manage Identity and Access Management (IAM) on AWS.
Collaborate with stakeholders to understand data requirements and deliver effective solutions.

Required Skills & Experience:

8-10 years of experience as a Data Engineer or Developer.
Proven experience building and optimizing data pipelines on AWS.
Proficiency in scripting with Python.
Strong working knowledge of:
Big Data Tools: AWS Athena.
Relational & NoSQL Databases: AWS Redshift and PostgreSQL.
Data Pipeline Tools: AWS Glue, AWS Data Pipeline, or AWS Lake Formation.
Container Orchestration: Kubernetes, Docker, Amazon ECR/ECS/EKS.
Experience with wrangling, exploring, and analyzing data.
Strong organizational and problem-solving skills.

Preferred Skills:

Experience with machine learning tools (SageMaker, TensorFlow).
Working knowledge of stream processing (Kinesis, Spark-Streaming).
Experience with analytics and visualization tools (Tableau, Power BI).
Knowledge of optimizing AWS Redshift performance.

Education

Bachelor’s or Master’s Degree in Information Technology, Computer Science, or a relevant field.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.

Dice Id: 91166119
Position Id: 8915199
Posted 9 hours ago

Create job alert

Never miss an opportunity! Create an alert based on the job you applied for.

Similar Jobs

Data Engineer with Amazon client Exp

Santa Clara, California

•

8d ago

Hello,Greetings from Clifyx.Title: Data EngineerLocation: Santa Clara CA (Hybrid-Onsite)12+ Month ContractRelevant ExperienceMust have solid understanding of AWS and worked for Amazon in a similar role.Mandatory skillsAWS Data PlatformJob detailsData Engineer 7+ years of experience indata engineeringor related fields.Strong handson experience with:AWS services:Glue, S3, Redshift, EMR, Lambda, Kinesis, Athena.Big Data tech:Spark/PySpark, Hadoop, Hive.Programming:Python, SQL, Scala (optional).Data

Easy Apply

Contract

Depends on Experience

Senior AWS Developer

Remote

•

11d ago

Role: Senior AWS Developer Location: Remote AWS Developer role builds real-time applications processing a huge volume of data using AWS Services like Glue, Redshift, and Step Functions Responsibilities: Gather and document business requirements.Work with customers to identify the best solution for meeting business requirements.Design and build scalable ETL/ELT pipelines using AWS GlueAutomate and orchestrate workflows using AWS Step Functions, Glue Workflows, and Lambda.Develop and maintain da

Easy Apply

Contract

Depends on Experience

Lead Data Engineer with Full Stack Development

Remote

•

Yesterday

Pay Range: $60 - $85/hr. The pay rate may differ depending on your skills, education, experience, and other qualifications. Featured Benefits: Medical Insurance in compliance with the ACA.401(k).Sick leave in compliance with applicable state, federal, and local laws.Must Have AWS Dynamo, AWS Lambda, AWS - S3, AWS - SNS/SQS, AWS Redshift, AWS Athena, Apache Iceberg, Airflow/AWS Step Functions, AWS EMR, AWS Glue, AWS - Quick Suite/Tableau, AWS SageMaker Studio, AWS Lake Formation, Python, Ty

Easy Apply

Contract

70 - 85

Sr. Manager, Data & ML Engineering

Remote or California

•

Today

At Deckers Brands, Together, Every Step is a promise kept that every employee can bring their authentic self, is valued and supported, as a whole person, at work and beyond. Together, Every Step is how we continue to deliver exceptional business results, experience an amazing place to work, and have a positive impact on the communities and world around us. Job Title: Sr. Manager, Data & ML Engineering Reports to: Director, Data Platforms & Engineering Location: United States (Remote) Interest

Full-time

USD 170,000.00 - 190,000.00 per year

Search all similar jobs