Big Data \/ AWS Data Engineer

Note: This is a hybrid work. So the resource should either in Memphis, TN or in Virginia (Close to Hilton's McLean office) and be ready to go to either of these client locations three days a week.

We are seeking an experienced technical resource who could design, develop, support, and enhance enterprise data platforms while ensuring compliance with privacy and data governance requirements. The role combines data engineering, systems integration, analytics, solution design, and operational support.

POSITION GENERAL DUTIES AND TASKS :

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using Spark (Scala/Python), AWS Glue, AWS Lambda, and Shell scripting.
• Build and support ETL/ELT processes across AWS platforms, including S3, Hive, Apache Iceberg, and Amazon Redshift.
• Develop and maintain Redshift SQL/PLSQL code, stored procedures, and data-processing frameworks.
• Perform query tuning, performance optimization, and data-sharing implementations.
• Support ingestion and processing of structured and semi-structured data formats including JSON, CSV, XML, ORC, Parquet, and Avro.
• Design, enhance, and support solutions that provide a unified view of customer data across systems.
• Integrate multiple data sources, applications, and business processes to support reporting, analytics, and customer engagement initiatives.
• Collaborate with business and technical teams to define and implement scalable data solutions.
• Manage and support Data Subject Requests (DSRs), including customer data erasure requests.
• Work closely with the DPO team to ensure compliance with privacy regulations and internal policies.
• Maintain and support the MicroStrategy Privacy Compliance module. (Very infrequent once in year or 2 years)
• Contribute to data governance initiatives that improve data quality, consistency, and compliance.
• Analyze and troubleshoot complex data issues using Redshift SQL, MySQL, Python/PySpark, Shell scripting, and Excel.
• Investigate data mismatches, reporting issues, and business data anomalies.
• Support business analysis related to customer tiers, loyalty programs, and email marketability.
• Provide production support and root-cause analysis for critical data platform issues.

Technical Leadership & Collaboration

• Participate in Customer 360, Privacy, Data Governance, and Phaedon initiatives.
• Conduct code reviews and design reviews.
• Provide cross-team technical support and problem resolution.
• Research, design, and develop next-generation data capabilities and products.
• Act as a subject matter expert for customer data platforms.

Primary Technologies

• AWS: S3, EC2, Redshift (RA3/Serverless), S3 Tables, S3 Hive, AWS Glue, Lambda (Python), Kafka, Airflow, Apache Iceberg
• Spark (Scala/Python)
• Redshift SQL/PLSQL, MySQL
• Data Modeling and Query Optimization

Key Strengths

• Strong ownership and accountability
• Strong AWS Data Engineering stack awareness
• Ability to bridge business and technical teams
• Expertise across engineering, analytics, compliance, architecture, and operational support
• Adaptability and willingness to take on diverse data-related challenges

Required Qualifications

• 12+ years of experience in Data Warehouse / Big Data ecosystems.
• 5+ years in a lead /SME role
• Strong understanding of:
  • Data architecture & dimensional modeling
  • ETL/ELT frameworks
  • Distributed data processing
  • Cloud data platforms
  • Data governance and security
• Experience managing large-scale enterprise data environments.
• Proven track record of delivering complex data programs.
• Excellent communication and stakeholder management abilities.

Preferred Qualifications

• Experience in large enterprise or global delivery environments.
• Exposure to data modernization and cloud transformation programs.
• Certifications in cloud platforms (AWS/Azure/Google Cloud Platform) or data engineering.
• Experience in regulated industries (Hospitality, Finance, Healthcare, Telecom, etc.).

Key Competencies

• Strategic thinking with hands-on technical depth
• Executive presence and client-facing confidence
• Problem-solving and decision-making ability
• Operational excellence mindset