Amazon S3 Engineer

Location: Charlotte, NC
Salary: $120,000.00 USD Annually - $130,000.00 USD Annually
Description:
Job Title: Senior Amazon Data Engineer

Experience: 10+ Years
Location: Charlotte, NC or Plano, TX (Onsite)

Employment: Fulltime

Job Summary

We are seeking an experienced Senior Amazon Data Engineer to design, develop, and optimize large-scale data pipelines within a cloud-based data platform. The ideal candidate will have strong expertise in Amazon S3, AWS Data Engineering, PySpark, SQL, and ETL/ELT frameworks, along with hands-on experience in banking and payments data domains.

This role involves building reliable, scalable data solutions, ensuring high data quality and integrity, and working closely with cross-functional teams in a Cloudera-based big data environment.

Key Responsibilities

• Design, develop, and execute end-to-end data pipelines and validation test cases to ensure data accuracy, integrity, and quality.
• Build, implement, and optimize ETL/ELT workflows leveraging Amazon S3 for scalable data storage, retrieval, and processing.
• Develop scripts and automation for data movement between Amazon S3 and on-prem or cloud platforms using PySpark, Shell scripting, or similar tools.
• Utilize Amazon S3 advanced features such as:
• Versioning
• Lifecycle policies
• Access control (IAM policies, bucket policies)
• Server-side encryption
• Monitor, troubleshoot, and optimize pipeline performance to ensure high availability and reliability.
• Collaborate with data engineers, developers, analysts, and business stakeholders to resolve data-related issues efficiently.
• Validate and manage HiveQL, HDFS file structures, and data processing within Hadoop/Spark environments.
• Work within a metadata-driven ETL framework and batch/job scheduling systems.
• Create and maintain technical documentation, ensuring adherence to data engineering best practices.
• Support data governance, data quality checks, and compliance requirements within a regulated banking environment.

Required Skills & Qualifications

Technical Skills

• Primary Skill: Amazon Data Engineering
• Secondary Skills:
• AWS Data Engineering
• Amazon S3 (storage, lifecycle management, security, automation)
• PySpark
• SQL
• Shell Scripting
• Autosys (job scheduling)
• Strong hands-on experience with Hadoop, Spark, Hive, and HDFS
• Experience working with Oracle databases
• Strong understanding of data warehousing concepts, data transformation, and data quality frameworks
• Experience in Cloud Storage and AWS ecosystems

Tools & Platforms

• Amazon S3
• PySpark
• SQL
• Oracle
• Autosys
• Cloudera Platform

Domain Experience

• Banking domain experience is required
• Payments domain experience is preferred

Soft Skills

• Strong analytical and problem-solving skills
• Excellent verbal and written communication skills
• Ability to collaborate effectively with cross-functional and offshore teams
• Detail-oriented with a strong focus on data accuracy and quality

Preferred Qualifications

• Experience with metadata-driven ETL processes
• Exposure to large-scale distributed data processing
• Familiarity with enterprise data governance and compliance standards

By providing your phone number, you consent to: (1) receive automated text messages and calls from the Judge Group, Inc. and its affiliates (collectively "Judge") to such phone number regarding job opportunities, your job application, and for other related purposes. Message & data rates apply and message frequency may vary. Consistent with Judge's Privacy Policy, information obtained from your consent will not be shared with third parties for marketing/promotional purposes. Reply STOP to opt out of receiving telephone calls and text messages from Judge and HELP for help.

Contact:

This job and many more are available through The Judge Group. Please apply with us today!