Overview
Remote
Depends on Experience
Contract - W2
Contract - Independent
Contract - 6 Month(s)
Skills
Amazon Web Services
Apache Flume
Apache HBase
Apache Hadoop
Apache Hive
Apache Kafka
Apache NiFi
Apache Oozie
Apache Solr
Apache Spark
Cloud Computing
Big Data
Extract
Transform
Load
Health Care
Disaster Recovery
Data Migration
Cloudera Impala
Cloudera
MongoDB
Migration
Kerberos
Hue
Job Details
Title: Lead Big Data Cloud Engineer
Location: Remote
Terms of Employment:
W2 Contract-to-Hire, Six Months
This position is primarily remote. Candidates must be willing to work onsite in Reston, VA or Washington, DC once per month for all-hands meetings.
Overview & Responsibilities:
We are seeking a Lead Big Data Cloud Engineer for a crucial production support role with a leading healthcare services client. You will be responsible for the administration and support of the "Enterprise Data Hub" (EDH), a complex Big Data platform built on Cloudera (CDP) and running in the AWS cloud. This is a hands-on role for a lead engineer experienced with the Cloudera ecosystem (including Kafka, Nifi, Hbase, and Spark) who will ensure the health, performance, and security of 15 different data products and real-time data pipelines. You will
Provide lead-level production support for the Cloudera (CDP) platform running on AWS.
Monitor the health of all Hadoop daemon services and Cloudera services using Cloudera Manager.
Administer and troubleshoot the real-time data pipeline, including Apache Kafka (brokers, topics, streams) and Apache Nifi (flow management, registry).
Support and troubleshoot other ecosystem components, including Hbase and Solr.
Write and maintain shell scripts for health checks and monitoring.
Support data migrations and perform incremental updates/upgrades to the Cloudera environment.
Manage job workflows (Oozie, Hue) and implement security policies (Ranger).
Participate in a 24/7 on-call rotation to support the production platform.
Required Qualifications:
Must be available for a mandatory in-person interview in Reston, VA, or Washington, D.C.
Strong experience supporting Cloudera applications (CDP preferred) running in a Cloud (AWS) environment.
Strong administration and troubleshooting skills with Apache Kafka.
Strong administration and troubleshooting skills with Apache Nifi.
Experience supporting Hbase and Solr.
Experience writing Hive and Impala queries.
Experience with shell scripting for monitoring and automation.
Experience in a 24/7 production support role with on-call rotations.
Preferred Qualifications:
Experience with Cloudera migrations (CDH to CDP).
Experience with Mongo DB.
Experience with Cloud Disaster Recovery strategies.
Experience with Flume, Kudu, or Spark.
Experience implementing security with Kerberos and Ranger.
W2 Contract-to-Hire, Six Months
This position is primarily remote. Candidates must be willing to work onsite in Reston, VA or Washington, DC once per month for all-hands meetings.
Overview & Responsibilities:
We are seeking a Lead Big Data Cloud Engineer for a crucial production support role with a leading healthcare services client. You will be responsible for the administration and support of the "Enterprise Data Hub" (EDH), a complex Big Data platform built on Cloudera (CDP) and running in the AWS cloud. This is a hands-on role for a lead engineer experienced with the Cloudera ecosystem (including Kafka, Nifi, Hbase, and Spark) who will ensure the health, performance, and security of 15 different data products and real-time data pipelines. You will
Provide lead-level production support for the Cloudera (CDP) platform running on AWS.
Monitor the health of all Hadoop daemon services and Cloudera services using Cloudera Manager.
Administer and troubleshoot the real-time data pipeline, including Apache Kafka (brokers, topics, streams) and Apache Nifi (flow management, registry).
Support and troubleshoot other ecosystem components, including Hbase and Solr.
Write and maintain shell scripts for health checks and monitoring.
Support data migrations and perform incremental updates/upgrades to the Cloudera environment.
Manage job workflows (Oozie, Hue) and implement security policies (Ranger).
Participate in a 24/7 on-call rotation to support the production platform.
Required Qualifications:
Must be available for a mandatory in-person interview in Reston, VA, or Washington, D.C.
Strong experience supporting Cloudera applications (CDP preferred) running in a Cloud (AWS) environment.
Strong administration and troubleshooting skills with Apache Kafka.
Strong administration and troubleshooting skills with Apache Nifi.
Experience supporting Hbase and Solr.
Experience writing Hive and Impala queries.
Experience with shell scripting for monitoring and automation.
Experience in a 24/7 production support role with on-call rotations.
Preferred Qualifications:
Experience with Cloudera migrations (CDH to CDP).
Experience with Mongo DB.
Experience with Cloud Disaster Recovery strategies.
Experience with Flume, Kudu, or Spark.
Experience implementing security with Kerberos and Ranger.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.