AWS Data Engineering

Location: United States Experience: 8+ years Job Type: Permanent

### Top Skills & Technologies AWS | Redshift | Airflow | Kafka | SQL | Python | Data Modeling | Data Quality

### Job Summary Thinking Minds' client is seeking a senior data engineer to shape the data foundation for product analytics, personalization, and AI initiatives. This role owns event infrastructure from instrumentation through streaming and batch ingestion to warehouse delivery, while building reliable datasets and features for production machine learning. You will collaborate with product engineering, data science, ML engineering, analytics, and experimentation teams in an in-office environment.

### Ideal Candidate Should Have
Strong ownership of production data platforms, sound architectural judgment, and the ability to translate ambiguous business needs into practical designs.

Clear communication, collaborative technical leadership, and a commitment to mentoring, documentation, and engineering quality.

A proactive approach to evolving priorities and an interest in AI-enabled product experiences.

### Key Responsibilities
Build scalable pipelines on AWS using Airflow (MWAA), Glue, Kafka, S3, and Redshift; evolve event schemas, taxonomy, versioning, and migrations.

Establish Data Quality and observability through validation, instrumentation testing, anomaly detection, freshness monitoring, completeness checks, and lineage.

Handle late-arriving and out-of-order events with lookback reprocessing and idempotent incremental loads.

Develop personalization datasets and feature pipelines, including a batch and low-latency Feature Store connecting Redshift or S3 with DynamoDB.

Support AWS Batch and SageMaker workflows with training datasets, offline evaluation data, and model outputs delivered to downstream APIs.

Apply dimensional and event Data Modeling for BI, exploration, and experimentation; optimize warehouse distribution, sorting, workload management, unload patterns, and costs.

Instrument AI interaction, generation, and feedback events; establish code review, testing, CI/CD, documentation, and on-call standards.

### Required Qualifications
8+ years in data engineering, including ownership of production pipelines at scale, with strong SQL and Python skills.

Deep hands-on experience with AWS services including S3, Glue, Redshift, DynamoDB, Lambda, and IAM.

Production Apache Airflow experience covering DAG design, dependencies, templating, alerting, and backfills.

Streaming ingestion experience with Kafka, Kinesis, SQS, or similar technologies, alongside clickstream analytics, observability frameworks, and production ML data flows.

### Preferred Qualifications
Experience with PySpark or Spark, recommendation systems, interaction logs, embeddings, candidate generation, ranking features, and Embedding pipelines.

Familiarity with MLOps tooling, model registries, experiment tracking, and orchestration.

Exposure to LLM and Generative AI applications, vector stores, retrieval pipelines, evaluation datasets, and feedback loops.

Experience with instrumentation governance, A/B testing, experiment metrics, Iceberg, Delta, Hudi, or Redshift data sharing.

Thinking Minds' client offers architectural ownership, collaborative technical work, medical, dental, and vision coverage, a retirement-plan match, paid time off, tuition reimbursement, a lifestyle stipend, and relocation assistance.

#Dice