Hadoop / Big Data Platform Engineer

Job Summary (List Format):

- Serve as a subject matter expert (SME) in Hadoop/Big Data platform engineering, focusing on Cloudera Hadoop administration and related technologies (HDFS, YARN, Spark, Impala, Kafka, Linux/Unix).
- Manage and administer Hadoop clusters, including production support, troubleshooting, upgrades, monitoring, and performance tuning.
- Support NextGen Big Data platforms (Hadoop, Spark, Kafka, Impala, HBase, Docker, Ansible, etc.), including cluster management for Cloudera and Databricks.
- Provide end-to-end platform operations across all environments (upgrades, bug fixes, deployments, metrics/monitoring, disaster recovery, incident/problem/capacity management).
- Act as a technical liaison between clients, vendors, and project managers, offering solutions that meet user needs.
- Deliver L1/L2/L3 level support for the Hadoop platform and resolve issues related to job execution, NameNode, HDFS space, permissions, and backups.
- Analyze, debug, and troubleshoot Yarn, Spark, and Impala jobs; perform query and resource utilization tuning.
- Maintain strong technical knowledge of Hadoop architecture, relational databases, and data warehousing (SQL skills).
- Ensure integration with other applications and platforms, including authentication (AD/Kerberos), job scheduling (Autosys), and monitoring systems.
- Hands-on expertise with Cloudera Hadoop components (HDFS, Sentry, HBase, Kafka, Impala, SOLR, Hue, Spark, Hive, YARN, Zookeeper, Postgres).
- Utilize DevOps and automation tools (Ansible, Jenkins, SVN, Bitbucket) and scripting languages (Java, Python, Perl, Shell).
- Support ETL tools from the Hadoop stack and BI tools such as Tableau, including dashboard creation connecting Hive tables.
- Participate in agile development processes, collaborating with cross-functional teams at high technical levels.
- Preferred certifications: Cloudera Admin/Dev, Cloud, Docker-Container, OpenShift.
- Hybrid role based in Chicago, IL or Charlotte, NC (3 days onsite, 2 days remote); 4+ months contract.
- Shift timings: 8AM–5PM or 12:30PM ET–9:30PM ET.

Note: Strong experience with Kerberos/AD, Ansible, Jenkins, Docker/OpenShift, and Big Data technologies is highly preferred.