Cloudera Data Engineer

Job#: 3011123

Job Description:

Job Summary

Apex Systems is seeking a Cloudera Data Engineer to support the migration of our customers' Medicaid Data Warehouse Implementation in AWS environment. The resource will support the migration and continued operations of an existing Cloudera/Hive/Scala-based data pipeline environment from one AWS account to another.

This position is responsible for ensuring a seamless transition, validating data integrity and job performance, and maintaining reliable daily operations post-migration.

The role will work closely with the existing project team for the underlying AWS infrastructure (VPC, IAM, S3, EC2, networking). The resource will focus on Cloudera cluster migration, data pipeline reconfiguration, and operational stability.

Key Responsibilities

• Replicate and configure existing Cloudera cluster (HDFS, YARN, Hive, Spark) in the new AWS account.
• Coordinate with project team to ensure proper infrastructure provisioning (EC2, security groups, IAM roles, and networking).
• Reconfigure cluster connectivity and job dependencies for the new environment.
• Migrate and validate metadata stores (Hive Metastore, job configs, dependencies).
• Validate job execution and data outputs for parity with existing environment.
• Deploy, test, and operate existing Hive, Spark (Scala) jobs post-migration.
• Maintain job schedules, dependencies, and runtime configurations.
• Monitor job performance, identify bottlenecks, and apply tuning or code-level optimizations.
• Troubleshoot failures and implement automated recovery or alerting where applicable.
• Monitor Cloudera Manager dashboards, cluster health, and resource utilization.
• Manage user roles and access within Cloudera environment.
• Implement periodic data cleanup, archiving, and housekeeping processes.
• Document configurations, migration steps, and operational runbooks.

Required Skills and Experience:

Bachelor's degree in computer science, Information Systems, or a related field.

7+ years of experience in data engineering or big data development

4+ years' experience with Cloudera platform (HDFS, YARN, Hive, Spark, Oozie)

Experience deploying and operating Cloudera workloads on AWS (EC2, S3, IAM, CloudWatch)

Strong proficiency in Scala, Java and HiveQL; Python or Bash scripting experience preferred

Strong proficiency in Apache Spark & Scala programming for data processing and transformation.

Hands on experience with Cloudera distribution of Hadoop.

Hands-on experience implementing business-rules processing using Drools.

Able to work with infrastructure, DevOps, and data governance teams in a multi-disciplinary environment.

Preferred Qualifications:

Candidates with Cloudera certification (e.g., CDP Data Engineer or Cloudera Administrator)

Experience with Cloudera version upgrades or AWS-to-AWS environment migrations.

Experience in public-sector or large enterprise data environments.

EEO Employer

Apex Systems is an equal opportunity employer. We do not discriminate or allow discrimination on the basis of race, color, religion, creed, sex (including pregnancy, childbirth, breastfeeding, or related medical conditions), age, sexual orientation, gender identity, national origin, ancestry, citizenship, genetic information, registered domestic partner status, marital status, disability, status as a crime victim, protected veteran status, political affiliation, union membership, or any other characteristic protected by law. Apex will consider qualified applicants with criminal histories in a manner consistent with the requirements of applicable law. If you have visited our website in search of information on employment opportunities or to apply for a position, and you require an accommodation in using our website for a search or application, please contact our Employee Services Department at or .

Apex Systems is a world-class IT services company that serves thousands of clients across the globe. When you join Apex, you become part of a team that values innovation, collaboration, and continuous learning. We offer quality career resources, training, certifications, development opportunities, and a comprehensive benefits package. Our commitment to excellence is reflected in many awards, including ClearlyRated's Best of Staffing in Talent Satisfaction in the United States and Great Place to Work in the United Kingdom and Mexico.

Apex Benefits Overview: Apex offers a range of supplemental benefits, including medical, dental, vision, life, disability, and other insurance plans that offer an optional layer of financial protection. We offer an ESPP (employee stock purchase program) and a 401K program which allows you to contribute typically within 30 days of starting, with a company match after 12 months of tenure. Apex also offers a HSA (Health Savings Account on the HDHP plan), a SupportLinc Employee Assistance Program (EAP) with up to 8 free counseling sessions, a corporate discount savings program and other discounts. In terms of professional development, Apex hosts an on-demand training program, provides access to certification prep and a library of technical and leadership courses/books/seminars once you have 6+ months of tenure, and certification discounts and other perks to associations that include CompTIA and IIBA. Apex has a dedicated customer service team for our Consultants that can address questions around benefits and other resources, as well as a certified Career Coach. You can access a full list of our benefits, programs, support teams and resources within our 'Welcome Packet' as well, which an Apex team member can provide.