Lead Sr Software Engineer(Cloudera Hadoop stack)

Duration: 6 Months (C2H)

Work Mode: 100% Remote

Interview Type: Virtual (2 Rounds)



Job Summary

Our client is a leading healthcare analytics and solutions company that leverages vast clinical and financial datasets to generate insights that improve financial performance, reduce operational inefficiencies, and enhance healthcare quality across the system.

We are seeking an experienced Senior Software Engineer to lead the design, development, and modernization of high-quality software applications and big data solutions. In this role, you will be pivotal in evaluating current on-premises Cloudera Hadoop workflows and driving a comprehensive migration strategy to native AWS cloud solutions. You will work closely with cross-functional partners across Infrastructure, Content teams, Product Owners, QA, and DevOps, serving as a technical lead and mentor.

Key Responsibilities

Design & Development: Build scalable, maintainable software modules and data-processing solutions for healthcare analytics applications in accordance with technical design and coding standards.

Cloud Migration Strategy: Analyze existing Cloudera Hadoop architecture, workflows, and pipelines (HDFS, Hive, Spark, Oozie, Python) and execute target migrations to cloud-native AWS environments.

Architecture & Tooling: Design AWS architectures utilizing services like Amazon EMR, AWS Glue, Apache Airflow, and AWS Lambda; select appropriate cloud tools for ingestion, processing, and analytics.

Implementation & Operations: Oversee data transfer, application re-platforming, validation, and testing; create operational runbooks, technical migration plans, and documentation.

Scripting & Automation: Write, maintain, and troubleshoot shell scripts (Bash) for automation, deployment, and operational support.

Stakeholder Collaboration: Manage communications and alignment with cross-functional stakeholders including Infrastructure, Content, QA, Product, and DevOps.

Quality & Security: Ensure overall solutions meet performance, scalability, data security, compliance, and governance benchmarks.

Mentorship & Standards: Conduct code/solution reviews and mentor junior and peer engineers in software engineering methods and AWS big data best practices.

Required Qualifications

Experience: 10+ years of software/data engineering experience applying enterprise design patterns to Big Data platforms in AWS.

Hadoop Ecosystem: Deep hands-on expertise with the Cloudera Hadoop stack (HDFS, Hive, Spark, Oozie, Python).

AWS Big Data: Demonstrated experience migrating legacy big data architectures to AWS cloud services (Amazon EMR, AWS Glue, Apache Airflow, AWS Lambda).

Platform Engineering: 5+ years of experience with build and deployment tools such as Bitbucket, Gradle, Octopus Deploy, and Artifactory.

Infrastructure as Code: Experience with cloud automation tools like Terraform.

Scripting & Formats: Proficiency in Bash/Shell scripting and working knowledge of XML and JSON formats.

Soft Skills: Excellent stakeholder management skills, clear communication, ability to manage migration scope, and proven mentoring capabilities.

Education: Bachelor s Degree in Computer Science, Information Technology, Information Systems, Engineering, or a related field.

Preferred Qualifications

Healthcare Domain: Knowledge of healthcare analytics, medical claims data, and relationships between providers and members.

Certifications: AWS Certified Solutions Architect or AWS Certified Big Data - Specialty.