Cloudera Hadoop / Spark / Airflow Admin

Overview

On Site
Accepts corp to corp applications
Contract - W2
Contract - 12
100% Travel

Skills

spark
Hadoop
Cloudera
Ldap
Airflow

Job Details

Cloudera Hadoop / Spark / Airflow Admin:

Administer Cloudera Hadoop clusters (HDFS, YARN, Hive, Impala) using Cloudera Manager; ensure high availability and performance.

Perform OS and platform patching/upgrades for Hadoop, Spark, and Airflow components with rollback plans.

Manage Spark workloads: tune executors, memory, and dynamic allocation for optimal performance.

Install, configure, and maintain Apache Airflow (Scheduler, Webserver, Workers); ensure DAG reliability and SLA compliance.

Implement security and compliance: Kerberos, LDAP/AD integration, Ranger policies, TLS certificates.

Monitor and troubleshoot platform health using Cloudera Manager, PrometheGrafana, and log analysis.

Automate operational tasks with Ansible, Python, and shell scripting for provisioning, patching, and config management.

Handle incident and change management: root cause analysis, documentation, and on-call support.

Perform capacity planning and resource optimization for multi-tenant clusters.

Collaborate with data engineering and infrastructure teams to ensure platform stability and scalability.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.