Search Jobs | Dice.com

Job Details

Skills

AWS
Python
Kafka
Glue
Architecture
Data Engineer

Summary

Job Title: Sr.Data Engineer AWS & Streaming

Location: Fort Mill SC or New York, NY (2-3 Days Hybrid)

Experience level 10- 15 Years

Role Summary:
We are seeking a Mid Senior Data Engineer with strong expertise in AWS-based data engineering, real-time streaming technologies, and enterprise-grade data quality frameworks. The ideal candidate will design, build, and optimize scalable batch and streaming data pipelines, implement robust data validation and monitoring processes, and support mission-critical analytics platforms.

Key Responsibilities:

Develop and maintain scalable ETL/ELT pipelines using AWS Glue, PySpark, and Python
Build event-driven workflows using AWS Lambda
Design and manage real-time streaming solutions using Kafka, KSQL, and Apache Flink
Implement and enforce comprehensive data quality frameworks, including validation, profiling, monitoring, and reconciliation
Optimize data processing performance, scalability, reliability, and cost in cloud environments
Collaborate with cross-functional teams to deliver reliable, production-grade data platforms and ensure data integrity across the pipeline

Required Skills:

Strong hands-on experience with Python and PySpark
Proven expertise in AWS Glue, Lambda, and other cloud-native data services
Solid experience with the Kafka ecosystem (topics, partitions, consumer groups, streaming patterns)
Demonstrated experience building and supporting data quality frameworks (validation rules, reconciliation checks, profiling, anomaly detection)
Strong understanding of distributed data processing and scalable architecture patterns

Good-to-Have Skills:

Experience with Apache Flink for real-time stream processing and stateful computations
Knowledge of KSQL or other streaming SQL engines
Exposure to CI/CD pipelines, IaC (Terraform/CloudFormation), and DevOps practices
Familiarity with data lake/lakehouse architectures and table formats such as Iceberg, Delta, or Hudi
Experience working in enterprise or financial data environments

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.

Dice Id: 90969158
Position Id: 2026-4926
Posted 1 day ago

Fitment

Dice Job Match Score™

🛠️ Calibrating flux capacitors...

Create job alert

Never miss an opportunity! Create an alert based on the job you applied for.

Similar Jobs

Data Engineer

Hybrid in Fort Mill, South Carolina

•

7d ago

Title: Data Architect Engineer Duration: 6 month contract (potential to extend or convert) Work Location: Charlotte, NC preferred. If remote, would need to come onsite once per month for 2 days once converted to full time. Job Description: The Data Architect Engineer will be responsible for designing and implementing robust, scalable, and high-performing data solutions on AWS. You will work closely with data engineers, software developers, and business stakeholders to ensure our cloud data infr

Easy Apply

Contract

$45 - $57

Data Architect Engineer (AWS / Python) - 3635382

Charlotte, North Carolina

•

Today

Lighthouse Technology Services is partnering with our client to fill their Data Architect Engineer (AWS / Python) position! This is a 6 months contract-to-hire opportunity and will be hybrid in Charlotte, NC. This role will be a W2 employee of Lighthouse Technology Services. No C2C or subcontracting arrangements will be considered. What You'll Be Doing: Design and implement cloud-native data architectures on AWS, including data lakes, data warehouses, and real-time data processing pipelines fo

Full-time

USD 65.00 - 72.00 per hour

*Data Architect* - W2 role

Remote or Hybrid in Charlotte, North Carolina

•

8d ago

Kellton Tech is a full-service software development company, offering end-to-end IT solutions, strategic technology consulting and product development services in Web, SMAC (Social, Mobile, Analytics, Cloud), ERP-BPM, and IoT space Our methodology of inventing infinite possibilities with technology helps us develop best in-class and cost effective solutions for our clients. Currently Kellton Tech is looking for talented resources for one of our listed client. Below are the position details Job T

Easy Apply

Contract

Depends on Experience

Data Engineer

Charlotte, North Carolina

•

28d ago

Data Engineer (PySpark) Short Job Description Role: Data Engineer PySpark Key Responsibilities: Develop and maintain ETL/data pipelines using PySpark.Process and transform large datasets efficiently.Write optimized SQL queries for data extraction and analysis.Build and maintain data lakes/data warehouses.Optimize Spark jobs for performance and scalability.Work with cloud platforms (AWS/Azure/Google Cloud Platform) and workflow tools like Airflow.Collaborate with data analysts, data scientists,

Easy Apply

Full-time

Depends on Experience

Filter Results

Job post features

Posted date

Work settings

Employment type

Distance

Employer type

Work authorization

Data Engineer AWS & Streaming

Skills

Summary

Dice Job Match Score™

Similar Jobs

Data Engineer AWS & Streaming

Job Details

Skills

Summary

Dice Job Match Score™

Similar Jobs