Hadoop Administrator

Hadoop/Spark cluster environments,HDFS, Hive, Pig and MapReduce
Contract Corp-To-Corp, Contract Independent, Contract W2, 12 Months+
Depends On Experience
Telecommuting not available Travel not required

Job Description

Location:  Downtown Chicago IL or Richardson, TX


• Short Description 
o We are seeking experienced and seasoned Hadoop Administrators, the primary responsibilities are: 
§ Manage large scale Hadoop/Spark cluster environments, handling all environment builds, including design, capacity planning, cluster setup, performance tuning, scaling (adding/removing nodes), scheduling and monitoring (including log reviews and job tracking/reporting), setting up Kerberos principals and testing HDFS, Hive, Pig and MapReduce access for the new users. 
§ Understanding of interactions between applications, HW, and UNIX/Linux operating system and to provide recommendations and suggestions related to troubleshooting and performance improvement 
§ Deploy Hadoop cluster(s), add and remove nodes, keep track of jobs, monitor critical parts of the cluster, configure name-node high availability, schedule and configure. 


• Minimum requirements 
o 2+ years of demonstrable experience as a Hadoop Administrator for mid-level candidates 
o 4+ years of demonstrable experience as a Hadoop Administrator for senior-level candidates 
o BS/equivalent or higher degree (Computer Science, Math or Engineering preferred) 
o 90% travel required for candidates flying to client premise 
o Work is 100% onsite at the customer location 


• Preferred requirements 
o OPTION 1 Hadoop Administrator Certification with Hortonworks distribution 
o Azure deployment experience 

Hadoop Administrator 
Minimum Job Responsibilities: 
1. Manage large scale Hadoop/Spark cluster environments, handling all environment builds, including design, capacity planning, cluster setup, performance tuning, scaling (adding/removing nodes), scheduling and monitoring (including log reviews and job tracking/reporting), setting up Kerberos principals and testing HDFS, Hive, Pig and MapReduce access for the new users. 
2. Understanding of interactions between applications, HW, and UNIX/Linux operating system and to provide recommendations and suggestions related to troubleshooting and performance improvement 
3. Deploy Hadoop cluster(s), add and remove nodes, keep track of jobs, monitor critical parts of the cluster, configure name-node high availability, schedule and configure. 
4. Implement automated approaches for system administration and operation tasks, provisioning new servers and deployment of artifacts for multiple applications. 
5. Evaluate and recommend systems software and hardware for the Big Data enterprise system, including collaborating with application teams to install operating system and Hadoop updates, patches, version upgrades when required. 
6. Contribute to the evolving architecture of our Big Data services to meet changing requirements for scaling, reliability, performance, manageability, and price. 
7. Develop metrics and measures of utilization and performance. 
8. Manage cluster resource to meet application jobs’ SLAs, including support of DR, Backup/Recovery and archiving operations) 
9. Handling job queues and workload mapping for multi-tenant environment with different level of priorities and SLAs. 
10. Management and creation of support documentation for standard procedures. 
11. Liaison with partner technology providers (ex: Cloudera, Hortonworks, Azure, AWS, etc…) on escalating, logging and resolving issues, managing enhancements, upgrades and patches. 
Core Skills/Experience (minimum): 
1. Professional experience supporting production Linux environments (2 years mid-level minimum; 4 years senior level minimum). 
2. Expert knowledge of Hadoop/Spark design principals, cluster connectivity, security (ex: Kerboros, LDAP and Active Directory integration) and the factors that affect distributed system performance.(3 years) 
3. Extensive experience and understanding across various stack components (HDFS, Map Reduce, Spark Core, Pig, Hive, Zookeeper, Mahout, Elastic Stack, workflow tools such as Oozie and Azkaban) 
4. Advanced hands-on Linux skill is a must (2 years mid-level; 4 years senior level minimum). Understanding of shell, debugging things etc. 
5. Experience with performance tuning, capacity planning, and workload mapping 
6. Experience with handling job queues for multi-tenant environment with different level of priorities and SLAs. 
7. Experience with complex networking infrastructure including firewalls, VLANs, and load balancers. 
8. Experience integrating third party tools with Hadoop such as Tableau, PowerBI, Jupyter Hub, Tensorflow, H2O.ai, etc. (1 years) 
9. Scripting Language experience (ex: Python, Ruby, Bash) (3 years) 
10. Experience in using automation and container tools such as Ansible, Docker, Jenkins, Puppet and Chef 
11. Manage content repositories and structures for support code, scripting and versioning. (ex: Github) 
12. DR, Backup/Recovery and archiving experience 
13. BS/equivalent or higher degree (Computer Science, Math or Engineering preferred) 

Value-add Skills: 
1. Experience working on NoSQL and Search technologies (Ex: Hbase, Mongo) 
2. Knowledge of best practices related to security, performance, and disaster recovery. 
3. Experience with monitoring frameworks like Ganglia and Nagios 
4. Experience in Healthcare 
5. Preferred: Hadoop Administrator Certifications 
6. Preferred: Experience in Cloud platforms 
7. Preferred: Experience in specific Cloud-based Hadoop deployments (AWS-EMR, Azure HDInsight, HDP or CDP on Cloud, etc) 
8. Container deployment/management experience 
9. Basic SQL and relational database experience 
10. Basic skills in ETL (data ingestion, transformation and harmonization, metadata ) 
11. Java/Scala experience 
12. Direct end-user/ business-client facing interaction experience 

 

 

 

Thank you for your time.

Priya Goel

US Tech Solutions

Tel: 2019320901

Email: priya@ustechsolutionsinc.com

URL: www.ustechsolutions.com

Dice Id : usts
Position Id : 886302
Have a Job? Post it

Similar Positions

Cassandra DBA
  • InfoVision, Inc.
  • Dallas, TX
Senior Hadoop Administrator
  • Epsilon
  • Irving, TX
Hadoop Consultant/Admin
  • Charter Global, Inc.
  • Alpharetta, GA
Kafka Administrator
  • Enable Data
  • Dallas, TX
Hadoop developer
  • Collabera
  • Irving, TX
Hadoop/Scala Developer
  • Mindlance
  • Plano, TX
Big Data/ Hadoop
  • CSI Consulting
  • Dallas, TX
BigData/Hadoop Architect with Azure- JP
  • Central Business Solutions
  • Plano, TX
RPA Developer
  • Pyramid Consulting, Inc.
  • Ashburn, VA
Bigdata Spark Developer
  • Orabase Solutions LLC.
  • Irving, TX