Hadoop

Big Data Hadoop/AWS

Job Description

Overview:

• Client is seeking a Senior Software Engineer to lead the design and development of software applications and reusable software components.
• Work closely with Senior Developers, QA, Product Owners, Project Managers, and other stakeholders.
• Develop scalable and innovative applications supporting healthcare analytics and data-driven solutions.
• Follow established coding standards, technical designs, and software engineering best practices.

Responsibilities

• Design and develop high-quality, maintainable software modules and applications.
• Develop reusable software components following established coding and architectural standards.
• Perform unit testing and integration testing using appropriate methodologies and techniques.
• Analyze business requirements and technical specifications and create detailed implementation designs.
• Analyze, troubleshoot, and resolve software-related issues reported by internal or external customers.
• Stay current with emerging technologies and contribute to their adoption within the product suite.
• Follow change-control processes and maintain accurate technical and operational information.
• Review proposed software engineering solutions to ensure adherence to best practices.
• Mentor team members on software engineering methodologies, development practices, and technical standards.
• Collaborate effectively with developers, QA teams, product owners, project management, and business stakeholders.
• Complete assigned special projects and other responsibilities as required.
• Perform all assigned duties with or without reasonable accommodation.

Required Qualifications

• Bachelor s degree in Computer Science, Information Technology, Information Systems, Engineering, or a related field.
• 10+ years of experience with enterprise and application design patterns.
• Strong experience with Hadoop and Big Data platforms on AWS.
• Deep expertise in both on-premises Hadoop ecosystems and modern cloud-native Big Data solutions on AWS.
• Hands-on experience with Cloudera Hadoop stack, including:
  • HDFS
  • Hive
  • Spark
  • Oozie
  • Python
• Strong proficiency in Shell/Bash scripting for automation and operational support.

AWS / Hadoop Migration Experience

• Analyze existing Cloudera Hadoop architecture, workflows, and data pipelines.
• Develop comprehensive migration strategies for moving Hadoop workloads to AWS.
• Migrate data, processing logic, applications, and workflows from on-premises Hadoop to AWS.
• Design target architectures using AWS Big Data services such as:
  • Amazon EMR
  • Apache Airflow
  • AWS Glue
  • AWS Lambda
• Evaluate and select appropriate AWS services for:
  • Data ingestion
  • Data storage
  • Data processing
  • Data analytics
• Lead migration activities including:
  • Data transfer
  • Application re-platforming
  • Data validation
  • Performance validation
• Ensure migrated architecture meets requirements for:
  • Performance
  • Scalability
  • Reliability
  • Security
• Collaborate with developers, data engineers, DevOps teams, and business stakeholders.
• Create detailed technical documentation and migration runbooks.
• Mentor team members on AWS Big Data best practices.

Cloud / Infrastructure Requirements

• Strong understanding of cloud data security, compliance, and governance.
• Proven experience migrating or modernizing Big Data workloads to AWS.
• Experience with Cloud Automation and Infrastructure as Code (IaC).
• Hands-on experience with Terraform or similar IaC tools.
• AWS certifications such as AWS Certified Solutions Architect or AWS Certified Big Data Specialty are highly desirable.

Additional Required Skills

• Familiarity with XML and JSON data formats.
• Knowledge of monitoring and logging tools.
• 5+ years of experience with platform engineering tools, such as:
  • Bitbucket
  • Gradle
  • Octopus Deploy
  • Artifactory

Preferred Qualifications

• Bachelor s degree in Computer Science, Engineering, or a related technical field.
• Knowledge of healthcare data, particularly:
  • Healthcare claims
  • Providers
  • Members
  • Relationships between providers, members, and claims
• Experience working with healthcare analytics or healthcare data platforms is a plus.