AWS Data Platform / Platform Engineering Lead
Location: Irvine, CA
Work Arrangement: Onsite – 5 Days/Week
Rate: $70–$72/hr on W2
Client: TCS / Capital Group
Job Summary
We are seeking a highly experienced AWS Data Platform / Platform Engineering Lead to design, build, automate, and support enterprise-scale cloud data platforms and DevOps infrastructure.
The ideal candidate will have deep hands-on experience with AWS, Data Platform Engineering, Platform Engineering, Jenkins, CI/CD, Terraform, SRE, Infrastructure as Code (IaC), GitHub, SonarQube, ROC/D, and cloud-based DevOps.
This role will be responsible for building scalable and secure AWS data-platform infrastructure, developing automated CI/CD pipelines, implementing infrastructure automation, establishing code-quality controls, and improving platform reliability and operational excellence.
The candidate should be comfortable working across cloud infrastructure, data platforms, DevOps automation, CI/CD pipelines, infrastructure provisioning, deployment automation, and platform operations.
Key Responsibilities
AWS Cloud & Data Platform
- Architect, build, and manage enterprise AWS data platforms supporting data engineering, analytics, reporting, and business-critical workloads.
- Design scalable and highly available cloud-native data platform architectures.
- Build and manage AWS infrastructure supporting data ingestion, processing, storage, transformation, and analytics.
- Work extensively with AWS services such as S3, Glue, Redshift, Athena, Lambda, EMR, EKS, EC2, IAM, VPC, CloudWatch, KMS, and Secrets Manager.
- Develop secure and reusable cloud infrastructure patterns for data engineering and analytics teams.
- Implement AWS security, IAM, networking, encryption, access control, and governance.
- Support data-platform scalability, reliability, performance, availability, and cost optimization.
- Implement monitoring, logging, alerting, and observability for AWS data-platform environments.
- Collaborate with Data Engineering, Architecture, Security, Application, and Infrastructure teams.
Data Platform Engineering
- Build and support modern enterprise data-platform infrastructure.
- Support Data Lake / Data Lakehouse architectures and cloud-based data workloads.
- Enable data ingestion, batch processing, streaming, transformation, and analytical workloads.
- Support technologies such as Databricks, Spark/PySpark, dbt, Airflow, AWS Glue, and Redshift.
- Develop standardized infrastructure and deployment patterns for data engineering teams.
- Implement data-platform security, governance, monitoring, availability, and operational standards.
- Troubleshoot infrastructure and platform issues impacting data pipelines and analytics workloads.
Jenkins / CI/CD
- Design, develop, and maintain enterprise CI/CD workflows using Jenkins Pipelines.
- Build automated pipelines for source control, build, testing, quality validation, infrastructure provisioning, deployment, and release management.
- Install, configure, and maintain Jenkins plugins required for Git/GitHub repositories and enterprise CI/CD workflows.
- Configure SCM Polling, Git Webhooks, automated triggers, and pipeline orchestration.
- Develop reusable Jenkins pipeline frameworks and Shared Libraries.
- Integrate Jenkins with GitHub, Terraform, SonarQube, AWS, and other DevOps tools.
- Troubleshoot Jenkins pipeline failures, deployment issues, build failures, and integration problems.
SonarQube / Quality Gates
- Integrate SonarQube into Jenkins CI/CD pipelines.
- Configure and enforce SonarQube Quality Gates.
- Automate code-quality and security validation within CI/CD workflows.
- Ensure code and deployments meet defined enterprise quality standards.
- Monitor and resolve quality-gate failures before application or platform deployment.
Terraform / Infrastructure as Code
- Design and implement AWS infrastructure using Terraform.
- Develop reusable and standardized Terraform modules.
- Automate provisioning and configuration of AWS infrastructure and data-platform components.
- Integrate Terraform with Jenkins CI/CD pipelines.
- Implement automated Terraform plan, validation, approval, and deployment workflows.
- Manage infrastructure changes through Git-based version control.
- Establish enterprise standards for Infrastructure as Code, automation, security, and governance.
ROC/D & DevOps Automation
- Implement and support ROC/D within enterprise DevOps and CI/CD workflows.
- Integrate ROC/D with applicable source-control, pipeline, deployment, and infrastructure automation processes.
- Support automated release, deployment, and operational workflows involving ROC/D.
- Troubleshoot ROC/D-related deployment, pipeline, and platform issues.
- Work with engineering teams to standardize and automate release and operational processes.
Platform Engineering
- Establish enterprise Platform Engineering standards, automation frameworks, and reusable platform capabilities.
- Build self-service infrastructure and deployment capabilities for engineering and data teams.
- Standardize development, testing, and production deployment processes.
- Promote automation, GitOps, CI/CD, IaC, observability, security, and platform reliability.
- Reduce manual infrastructure and deployment activities through automation.
- Establish operational readiness, monitoring, incident management, and reliability practices.
- Continuously improve platform scalability, availability, security, and engineering productivity.
Required Skills
Mandatory Technical Skills
- AWS Cloud – Deep hands-on experience
- AWS Data Platform Engineering – Deep experience
- Platform Engineering
- Jenkins / Jenkins Pipelines
- CI/CD
- Terraform
- Infrastructure as Code (IaC)
- Git / GitHub
- SonarQube / Quality Gates
ROC/D
SRE
- Cloud DevOps
- Infrastructure Automation
- Monitoring & Observability
- Python, Bash, or Shell scripting
AWS Skills
Strong hands-on experience with multiple AWS services:
S3 | AWS Glue | Redshift | Athena | Lambda | EMR | EKS | EC2 | IAM | VPC | CloudWatch | KMS | Secrets Manager
Candidate must understand how AWS services are integrated to create and operate enterprise data platforms.
Data Platform Skills
Strong understanding of:
- Data Lake / Data Lakehouse
- Enterprise Data Platforms
- Data ingestion and integration
- Batch and streaming data processing
- Data pipelines
- Data transformation
- Data storage and analytics
- Data platform security
- Data governance
- Data quality
- Platform monitoring and observability
- High availability and disaster recovery
- Data-platform performance and scalability
Databricks, Spark/PySpark, dbt, Airflow, AWS Glue, and Redshift experience is highly preferred.
Preferred Skills
- Databricks
- Apache Spark / PySpark
- dbt
- Apache Airflow
- AWS Glue
- Amazon Redshift
- Amazon EMR
- Kubernetes / Amazon EKS
- Docker
- GitOps
- Jenkins Shared Libraries
- Terraform Enterprise / Terraform Cloud
- Python
- Bash/Shell scripting
- Cloud security and governance
- Observability and monitoring
- Financial Services / Investment Management experience
Ideal Candidate Profile
The ideal candidate should be a hands-on Platform/Data Platform Engineer or Lead, not simply a traditional DevOps Engineer.
The candidate should demonstrate strong experience across:
AWS Cloud + Data Platform + Platform Engineering + Terraform/IaC + Jenkins CI/CD + GitHub + SonarQube + ROC/D + DevOps
They should be able to design the AWS data-platform infrastructure, automate it using Terraform, build Jenkins CI/CD pipelines, implement SonarQube quality gates, support ROC/D workflows, and operate the platform in an enterprise production environment.
Never miss an opportunity! Create an alert based on the job you applied for.
