Hadoop Platform Engineer

Skills:

On-prem instance

Hadoop config, performance, tuning

Ability to manage very large clusters and understand scalability

Interfacing with multiple teams

Many teams have self service capabilities, so should have this experience managing this with multiple teams across large clusters.Hands-on and strong understanding of Hadoop architecture

Experience with Hadoop ecosystem components - HDFS, YARN, MapReduce & cluster management tools like Ambari or Cloudera Manager and related technologies.

Proficiency in scripting, Linux system administration, networking, and troubleshooting skills

Qualifications:

• Bachelor's degree in Computer Science, Information Technology, or a related field (or equivalent work experience).
• Strong experience in designing, implementing, and administering Hadoop clusters in a production environment.
• Proficiency in Hadoop ecosystem components such as HDFS, YARN, MapReduce, Hive, Spark, and HBase.
• Experience with cluster management tools like Apache Ambari or Cloudera Manager.
• Solid understanding of Linux/Unix systems and networking concepts.
• Strong scripting skills (e.g., Bash, Python) for automation and troubleshooting.
• Knowledge of database concepts and SQL.
• Experience with data ingestion tools like Apache Kafka or Apache NiFi.
• Familiarity with data warehouse concepts and technologies.
• Understanding of security principles and experience implementing security measures in Hadoop clusters.
• Strong problem-solving and troubleshooting skills, with the ability to analyze and resolve complex issues.
• Excellent communication and collaboration skills to work effectively with cross-functional teams.
• Relevant certifications such as Cloudera Certified Administrator for Apache Hadoop (CCAH) or Hortonworks Certified Administrator (HCA) are a plus