Senior Big Data Engineer

Key Responsibilities

• Design and build scalable data lakes and data pipelines on AWS using cloud-native and automated approaches
• Enable federated and high-performance analytics using Amazon Athena and Trino, including query optimization at scale
• Manage metadata, schemas, and data discovery using AWS Glue Data Catalog
• Implement secure data governance using AWS Lake Formation, KMS encryption, and SSL
• Build, deploy, and operate data services on Kubernetes (Amazon EKS)
• Work with Hadoop ecosystem components (Spark, Hive, HDFS) including optimization techniques such as partitioning, bucketing, and columnar formats (Parquet, ORC)
• Troubleshoot and resolve complex issues across big data pipelines, clusters, and query engines
• Design and maintain CI/CD pipelines using Jenkins or similar tools
• Implement monitoring and observability using CloudWatch and Grafana
• Prepare curated, high-quality datasets for AI/ML use cases
• Build and configure MCP server for AI/ML integration
• Collaborate within Agile/Scrum teams and proactively identify architectural and performance gaps
• Propose scalable, innovative, and out-of-the-box engineering solutions

Required Qualifications

• 8+ years of experience in Big Data / Data Engineering
• Strong hands-on experience with AWS services:
  • S3, Glue Data Catalog, Lake Formation, Athena, EMR, EKS
• Strong experience with Trino (or Presto) and query optimization techniques
• Hands-on experience with Kubernetes (EKS) for data workloads
• Strong proficiency in SQL, Python, and shell scripting
• Experience building CI/CD pipelines using Jenkins or similar tools
• Experience building and configuring MCP servers for AI/ML integration
• Strong ownership mindset with problem-solving ability (not just task execution)

Preferred Qualifications

• Exposure to AI/ML data pipelines and workflows
• Experience in cloud-native data modernization programs
• Strong understanding of scalable data architecture in enterprise environments