Lead Data Engineer AWS

  • Fort Mill, SC
  • Posted 1 day ago | Updated 1 day ago

Overview

Hybrid
$120,000 - $140,000
Full Time

Skills

AWS Glue
AWS Redshift
Amazon S3
AWS Lambda
Amazon EMR
Amazon Kinesis
Amazon Athena
Python
SQL
ETL Development
Data Modeling
Data Warehousing
Terraform
AWS CloudFormation
CI/CD (Jenkins
CodePipeline)
Infrastructure as Code
AWS IAM
Data Governance
Data Security
CloudWatch
Big Data (Spark
Hadoop)
Monitoring
Automation
Leadership
Communication
Problem-Solving

Job Details

Job Summary:

We are seeking a highly skilled Lead Data Engineer with deep expertise in AWS cloud services to design, build, and optimize scalable data platforms. This role involves leading data engineering initiatives, architecting robust ETL pipelines, and collaborating with cross-functional teams to deliver secure, high-performance data solutions.

Key Responsibilities:

  • Architect and Develop Data Solutions:
    Design and implement large-scale data pipelines and data lakes using AWS services such as Glue, Redshift, S3, Lambda, EMR, Kinesis, and Athena.

  • Data Modeling & Warehousing:
    Build and optimize data models for analytics and reporting using Redshift and other AWS-native tools.

  • ETL Development:
    Develop and maintain ETL workflows leveraging AWS Glue, Step Functions, and Python.

  • Performance Optimization:
    Ensure data systems are highly available, secure, and optimized for performance and cost efficiency.

  • Automation & Infrastructure as Code:
    Implement automation using Terraform or CloudFormation for provisioning and managing AWS resources.

  • Data Governance & Security:
    Enforce data privacy, compliance, and security standards (e.g., IAM roles, encryption, GDPR).

  • Leadership & Mentorship:
    Lead a team of data engineers, provide technical guidance, and establish best practices for data engineering.

  • Collaboration:
    Work closely with data scientists, analysts, and business stakeholders to deliver data-driven solutions.

Required Skills & Qualifications:

  • Experience:
    8+ years in data engineering, with at least 3+ years in AWS-based solutions.

  • Technical Expertise:

    • AWS services: Glue, Redshift, S3, Lambda, EMR, Kinesis, Athena
    • Programming: Python, SQL
    • ETL tools and frameworks
    • Data modeling and warehousing concepts

  • Tools & Frameworks:

    • CI/CD pipelines (Jenkins, CodePipeline)
    • Infrastructure as Code (Terraform, CloudFormation)
    • Monitoring tools (CloudWatch)

  • Certifications (Preferred):
    AWS Certified Data Analytics Specialty or AWS Solutions Architect Professional.

  • Soft Skills:
    Strong leadership, communication, and problem-solving abilities.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.