Lead Data Engineer with AWS (Need 14+ Years profile & local to NC/SC)

Job Overview:

As a Lead Technical Consultant, you will participate in all aspects of the software development lifecycle, which includes estimating, technical design, implementation, documentation, testing, deployment, and support of applications developed for our clients. As a member working in a team environment, you will work with solution architects and developers on the interpretation/translation of wireframes and creative designs into functional requirements, and subsequently into technical design.

As an AWS Lead Data Engineer, you will participate in all aspects of the software development lifecycle, which includes estimating, technical design, implementation, documentation, testing, deployment, and support of applications developed for our clients.

Responsibilities:

• Design, develop, and maintain data pipelines using AWS services such as AWS Glue, AWS Lambda, Step Functions, Kinesis, and Data Pipeline.
• Implement and manage ETL/ELT processes, ensuring data quality, transformation, and orchestration.
• Work with structured and unstructured data sources, integrating data from multiple systems into data lakes, warehouses, and real-time streaming solutions.
• Optimize data workflows, performance tuning, and scalability using tools like Amazon Redshift, Athena, and S3.
• Collaborate with data scientists, analysts, and software engineers to enable efficient data accessibility and analytics.
• Implement data governance, security, and compliance best practices, including encryption, access control, and monitoring with AWS IAM, KMS, and CloudTrail.
• Automate data pipeline deployments using infrastructure as code (IaC) tools like Terraform and AWS CloudFormation.
• Monitor data reliability, integrity, and performance, leveraging tools such as AWS CloudWatch, Datadog, and Prometheus.
• Work in an Agile/Scrum environment, participating in sprint planning, reviews, and continuous improvement initiatives.

Qualifications:

• Passionate coders with 10+ years of application development experience.
• Self-management skills and effective communication in technical discussions
• Extensive experience with ETL processes.
• Deep expertise in AWS Glue, solid experience in Big Data, PySpark, and serverless.
• ETL and data streaming workflow experience.
• Strong in SQL queries and a deep understanding of data.
• Strong knowledge of Big Data concepts.
• Proficiency in SQL, including writing simple and complex queries, optimizing query performance, and familiarity with RANK functions.
• Ability to develop and maintain continuous data pipelines, ensuring data protection.
• Hands-on experience with Python for data-related development tasks.
• Client-facing or consulting experience is highly preferred.
• Skilled problem solvers with the desire and proven ability to create innovative solutions.
• Flexible and adaptable attitude, disciplined to manage multiple responsibilities and adjust to varied environments.
• Future technology leaders- dynamic individuals energized by fast-paced personal and professional growth.
• Phenomenal communicators who can explain and present concepts to technical and non-technical audiences alike, including high-level decision makers.
• Bachelor s Degree in MIS, Computer Science, Math, Engineering, or comparable major.
• Solid foundation in Computer Science, with strong competencies in data structures, algorithms, and software design.
• Knowledge and experience in developing software using agile methodologies.
• Proficient in authoring, editing, and presenting technical documents.
• Ability to communicate effectively via multiple channels (verbal, written, etc.) with technical and non-technical staff.