Job Title: Mid-Level Data Engineer
Location: Malvern, PA OR Charlotte, NC (Onsite/Hybrid as per project)
Employment Type: W2
Experience: 4 5 Years
Rate: $45 $50/hr (W2)
Job Summary
We are seeking a Mid-Level Data Engineer with 4 5 years of hands-on experience in building, maintaining, and optimizing data pipelines and data processing systems on AWS. The ideal candidate must have solid experience with AWS Glue, Lambda, Redshift, Athena, IAM, Python, Spark, and advanced SQL. This role will support large-scale data engineering initiatives for enterprise applications within TCS/Vanguard.
Key Responsibilities
Design, develop, and maintain ETL/ELT pipelines using AWS Glue, Lambda, and Python.
Build scalable data ingestion and transformation processes using Spark and distributed data processing techniques.
Develop and optimize queries in Amazon Redshift and Athena for analytics and reporting use cases.
Manage AWS resources and enforce IAM policies, ensuring security, data governance, and access control.
Work closely with data architects, analysts, and application teams to understand data needs and translate them into effective data solutions.
Perform data validation, quality checks, and implement best practices for data accuracy, reliability, and performance.
Monitor pipeline performance, troubleshoot failures, and optimize workloads for cost efficiency.
Participate in Agile ceremonies and contribute to sprint planning, estimation, and delivery.
Maintain documentation for data flows, data models, transformations, and operational processes.
Required Skills & Qualifications
4 5 years of hands-on experience as a Data Engineer.
Strong experience with AWS services including:
Glue (ETL jobs, Glue Catalog)
Lambda
Redshift (SQL, performance tuning)
Athena
IAM (roles, policies, permissions)
Proficiency in Python for data transformation and automation.
Hands-on experience with Apache Spark for distributed data processing.
Strong SQL skills with experience writing complex queries, optimizing execution plans, and handling large datasets.
Good understanding of data warehousing concepts, ETL/ELT lifecycles, and data lake architectures.
Experience working in Agile environments.
Strong communication skills and ability to collaborate with cross-functional teams.
Nice to Have
Experience with AWS Step Functions, S3 event-based workflows, or Glue Workflows.
Familiarity with CI/CD tools (CodePipeline, Jenkins, GitLab).
Exposure to version control (Git) and infrastructure as code (Terraform/CloudFormation).
Knowledge of data governance, security best practices, and compliance frameworks.
.jpg%3Fformat%3Dwebp&w=1080&q=75)
Never miss an opportunity! Create an alert based on the job you applied for.
