Lead AWS Data Engineer

Job Title: Lead AWS Data Engineer
Work Location: Jersey City, NJ(Onsite)
Duration :: Long Term Contract

Job Summary

We are seeking an experienced Lead AWS Data Engineer with 8+ years of experience designing, building, and optimizing scalable cloud-based data platforms. The ideal candidate will possess deep expertise in AWS data services, distributed data processing, ETL/ELT development, and data warehousing. This role requires strong technical leadership, hands-on development, and the ability to mentor engineers while collaborating with architects, business stakeholders, and cross-functional teams to deliver enterprise-grade data solutions.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using AWS cloud-native services.
• Build high-performance ETL/ELT workflows for structured, semi-structured, and unstructured data.
• Develop data ingestion frameworks for batch and real-time processing.
• Architect and optimize enterprise data lakes and data warehouse solutions.
• Implement data modeling techniques for analytical and reporting requirements.
• Lead technical design discussions and provide architectural guidance.
• Optimize data processing performance, scalability, and cost efficiency.
• Develop reusable frameworks, utilities, and automation for data engineering.
• Ensure data quality, governance, lineage, and security best practices.
• Collaborate with Data Scientists, BI Developers, Product Owners, and business stakeholders.
• Perform code reviews and mentor junior data engineers.
• Support CI/CD pipelines and infrastructure automation.
• Troubleshoot production issues and perform root cause analysis.
• Document technical solutions, architecture, and operational procedures.

Required Skills

• 8+ years of hands-on experience in Data Engineering.
• Strong expertise with AWS Cloud.
• Experience with AWS Glue, Amazon Redshift, Amazon S3, Lambda, EMR, Athena, Kinesis, RDS, Step Functions, CloudWatch, and IAM.
• Strong programming skills in Python and PySpark.
• Experience with Apache Spark for large-scale data processing.
• Advanced SQL development and query optimization.
• Experience building ETL/ELT pipelines.
• Hands-on experience with data lakes and modern data architectures.
• Experience with workflow orchestration tools such as Apache Airflow or AWS Managed Workflows for Apache Airflow (MWAA).
• Knowledge of streaming technologies such as Kafka or Amazon Kinesis.
• Experience implementing CI/CD using Git, Jenkins, GitHub Actions, or AWS CodePipeline.
• Familiarity with Infrastructure as Code using Terraform or AWS CloudFormation.
• Experience with REST APIs and data integration.
• Strong understanding of data security, encryption, and AWS best practices.
• Excellent debugging, analytical, and problem-solving skills.

Preferred Qualifications

• Experience with Databricks, Delta Lake, or modern lakehouse architecture.
• Knowledge of Snowflake or other cloud data warehouses.
• Experience with containerization technologies such as Docker and Kubernetes.
• Exposure to DevOps and MLOps practices.
• Experience with Data Catalog, Data Governance, and Metadata Management.
• AWS Certifications (Solutions Architect, Data Engineer, or Developer Associate/Professional).
• Experience working in Agile/Scrum environments.

Nice-to-Have Skills

• Terraform
• dbt
• Iceberg or Hudi
• Apache Kafka
• MongoDB
• NoSQL databases
• Power BI or Tableau
• Healthcare, Banking, Insurance, Retail, or Financial Services domain experience