Lead AWS Data Engineer

Role : Lead Data Engineer

Location: Newark, CA (Hybrid/Onsite)

Employment Type: Full-Time

Travel Requirements

While this is a fully remote position, occasional travel (up to 15% of the time) may be required for critical client meetings or team gatherings. All travel expenses will be covered by the company.

Key Responsibilities

? Solution Design & Architecture:

• Lead the design and architecture of robust and scalable data pipelines and workflows.


• Collaborate with customers to understand their requirements and translate them into technical solutions.


• Ensure solutions align with best practices for cloud-based data architectures (AWS preferred).

? Development & Implementation:

• Build and optimize data pipelines, ETL/ELT processes, and integrations with third-party systems.


• Implement data transformation and modeling strategies to support reporting, analytics, and machine learning initiatives.


• Implement robust data quality, standardization, and cleansing processes, particularly for PII data.


• Leverage cloud-native technologies such as AWS Glue, Lambda, S3, Redshift, DynamoDB and more.


• Design and implement data models, schemas, and data warehousing solutions.

Collaboration & Leadership:

• Serve as a technical lead for data engineering projects, mentoring junior engineers and providing technical guidance.


• Work closely with Project Managers, Solutions Architects, and DevOps teams to deliver end-to-end solutions.


• Lead requirement elicitation workshops with business SMEs to define data quality standards, and business rules.


• Engage with stakeholders to ensure solutions meet business objectives.

Quality Assurance:

• Implement and enforce coding standards, testing strategies, and CI/CD pipelines for data engineering workflows.


• Monitor and optimize performance and scalability of data solutions.

Customer Success:

• Act as a trusted advisor to customers, providing technical expertise during pre-sales and implementation phases.


• Ensure timely delivery of projects with high-quality outcomes.

Required Qualifications

• Bachelor s degree in Computer Science, Engineering, or a related field.


• 8+ years of experience in data engineering or related field, with at least 2 years in a senior or lead role.


• Expert knowledge of SQL and data modeling techniques.


• Strong proficiency in Python and/or Scala, SQL, and Spark.


• Hands-on experience with AWS data services (e.g., S3, Redshift, Glue, Athena, EMR, DynamoDB).


• Proven expertise in designing and implementing ETL/ELT pipelines and data integration solutions.


• Strong background in building and maintaining data pipelines at scale.


• Experience with data modeling, schema design, and working with relational and NoSQL databases.


• Demonstrated experience with Master Data Management (MDM) concepts and/or Entity Resolution techniques (e.g., matching algorithms, probabilistic vs. deterministic matching).


• Strong knowledge of data governance principles and implementing security controls for handling of PII/sensitive data.


• Knowledge of DevOps practices (e.g., CI/CD, infrastructure as code) and familiarity with tools like Terraform and Jenkins.


• Excellent communication skills, with the ability to collaborate across teams and present complex solutions to stakeholders.


• Proven track record of leading technical projects and mentoring teams.

Preferred Qualifications

• Master's degree in a relevant field.


• AWS certifications (e.g., AWS Certified Data Analytics - Specialty).


• Experience with modern data architectures, such as Data Lakes and Lake Houses.


• Familiarity with data visualization tools and platforms (e.g., Tableau, QuickSight, Power BI).


• Experience with real-time data processing and streaming technologies.


• Experience with specific MDM/ER tools (e.g., AWS Entity Resolution, Informatica, Reltio, or open-source matching libraries).


• Knowledge of machine learning workflows and their integration with data engineering pipelines.


• Knowledge of data governance and security best practices.


• Experience in consulting or professional services environments.


• Experience working on public sector (State/Government) projects.