Lead Cloudera AWS Support Engineer

Job Title: Lead Cloudera AWS Support Engineer/Hadoop Administrator
Location: Reston, VA (hybrid, onsite once a month)
Type: 6 month contract with possibility of extension
Compensation: $87/HR - $89/HR depending on experience

PURPOSE:
Performs complex analysis, design, development, automated unit and integration testing, and debugging of computer software ranging from operating system architecture integration and software design to selection of computer systems, language, and/or equipment. Designs, codes, tests, develops, and documents application programs for computer systems. Works with end-users and project stakeholders to define system scope and objectives. Manages modifications to and maintenance of operational programs and procedures. Leads stakeholder and product owner meetings to define software requirements and creates highly complex systems. Acts as a mentor to junior engineers

QUALIFICATIONS:

• Education: Bachelor's Degree in Information Technology or Computer Science
• Experience: 8 years Software design and development, software test and evaluation, software requirements management

SUMMARY:
Develop the scripts to monitor the services on top of the clusters and send out the alerts if necessary. Experience on the user incidents who have issues with the Cloudera Services and assisting them accordingly. Write the shell scripts to monitor the health check of Hadoop daemon services and respond accordingly to any warning or failure conditions. Monitoring the health of all the services running in the production cluster using the Cloudera Manager. Performing/Accessing the databases, metastore tables and writing Hive, Impala queries using HUE. Setting up the High Availability of the Services like Hue, Hive, HBase REST, SOLR and IMPALA on top of the all-new clusters that were build on the BDPaas Platform. Responsible for monitoring the health of the Services on top of all clusters. Working closely with different teams like Application development team, Security team, Platform Support to identify and implement the Configurational changes that are needed on top of the cluster for better performance of the services.

REQUIREMENTS:

• Experience in Supporting Cloudera Applications running in Cloud
• Experience in Supporting Cloud Applications with Mongo DB
• Experience in Cloud Disaster Recovery
• Apache Kafka - strong Administration & troubleshooting skills
• Experience with Kafka Streams API, Kafka integrations with MQ, and Kafka broker management
• Stream processing with KStreams & Ktables
• Topic/ offset management
• Apache Nifi - Administration, flow management, registry server management and controller service management
• Nifi to kafka /Hbase /solr integration
• Flume - integration with Kafka, Nifi & IBM MQ
• Hbase - administration, database management and troubleshooting
• Solr - administration
• Managing Logging level
• Managing shards & high availability
• Collection management
• Rectify resource intensive & long running solr queries

Additional Skills includes:

• Ensure Cloudera installation and configuration is at optimal specifications (CDP, CDSW, Hive, Spark, NiFi).
• Perform critical data migrations from CDH to CDP.
• Design and implement big data pipelines and automated data flows using Python/R and NiFi.
• Assist and provide expertise as it pertains to automating the entire project lifecycle.
• Perform incremental updates and upgrades to the Cloudera environment.
• Assist with new use cases (i.e., analytics/ML, data science, data ingest and processing), Infrastructure (including new cluster deployments, cluster migration, expansion, major upgrades, COOP/DR, and security).
• Assist in testing, governance, data quality, training, and documentation efforts.
• Move data and use YARN to allocate resources and schedule jobs.
• Manage job workflows with Oozie and Hue.
• Implement comprehensive security policies across the Hadoop cluster using Ranger.
• Configure and manage Cloudera Data Science Workbench using Cloudera Manager.
• Troubleshoot potential issues with Kerberos, TLS/SSL, Models, and Experiments, as well as other workload issues that data scientists might encounter once the application is running.
• Supporting the Big Data / Hadoop databases throughout the development and production lifecycle.
• Troubleshooting and resolving database integrity issues, performance issues, blocking and deadlocking issues, replication issues, log shipping issues, connectivity issues, security issues, performance tuning, query optimization, using monitoring and troubleshooting tools.
• Create, test, and implement scripting for automation support.
• Experience in working with Kafka ecosystem (Kafka Brokers, Connect, Zookeeper) in production is ideal
• Implement and support streaming technologies such as Kafka, Spark & Kudu

System One, and its subsidiaries including Joul, ALTA IT Services, and Mountain Ltd., are leaders in delivering outsourced services and workforce solutions across North America. We help clients get work done more efficiently and economically, without compromising quality. System One not only serves as a valued partner for our clients, but we offer eligible employees health and welfare benefits coverage options including medical, dental, vision, spending accounts, life insurance, voluntary plans, as well as participation in a 401(k) plan.

System One is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender identity, age, national origin, disability, family care or medical leave status, genetic information, veteran status, marital status, or any other characteristic protected by applicable federal, state, or local law.

#M-M2
#LI-RF1
#DI-RF1

Ref: #850-Rockville (ALTA IT)