Senior Data Engineer (AWS | Spark | Python)

Job Title: Senior Data Engineer (AWS | Spark | Python)
Location: Onsite 
Experience: 12+ Years (Strict)

Role Overview

We are seeking a high-caliber Data Engineer with deep expertise in building scalable, production-grade data platforms. This role is strictly for candidates who have hands-on experience designing and optimizing large-scale data pipelines in cloud environments.

You will work on high-impact data systems supporting analytics, AI/ML, and real-time decision-making. We are not looking for support-level or entry-level profiles—only engineers who have owned data systems end-to-end.

Must-Have Technical Skills (Non-Negotiable)

• Strong programming expertise in Python (production-level coding)
• Advanced SQL skills (query optimization, complex transformations)
• Hands-on experience with Apache Spark (batch and/or streaming)
• Deep experience with AWS:
• S3, Glue, Redshift, Lambda
• Building and maintaining ETL/ELT pipelines at scale
• Experience with data modeling (star/snowflake schemas)
• Working knowledge of distributed systems and large datasets

Preferred / High-Value Skills (Strong Advantage)

• Experience with Kafka or real-time data streaming
• Exposure to Snowflake / Databricks
• Knowledge of Airflow or workflow orchestration tools
• Experience supporting ML pipelines / MLOps workflows
• Infrastructure as Code (Terraform)

Core Responsibilities

• Design and build scalable, reliable data pipelines (batch & real-time)
• Optimize data workflows for performance, cost, and reliability
• Own end-to-end data architecture from ingestion to consumption
• Collaborate with Data Scientists, Analysts, and Engineering teams
• Ensure data quality, governance, and security standards
• Troubleshoot complex data issues in production environments

What We Are Looking For

• Engineers who have worked on production systems handling large-scale data
• Strong problem-solving mindset with system design capability
• Ability to write clean, efficient, and maintainable code
• Proven experience in performance tuning and optimization
• Clear communication and ownership mindset

Strict Screening Criteria (Important)

• Reject profiles with only:
  • Basic SQL or reporting experience
  • No real cloud (AWS) exposure
  • No hands-on Spark experience
• Prior experience in US-based projects or global clients is highly preferred

Nice-to-Have Industry Exposure

• Finance / Healthcare / E-commerce / SaaS data platforms