Overview
On Site
BASED ON EXPERIENCE
Contract - W2
Contract - Independent
Skills
Standard operating procedure
Google Cloud Platform
Use cases
Infrastructure management
High availability
Disaster recovery
RESTful
Development testing
Capacity management
Knowledge transfer
Apache Kafka
SAP HANA
Proxies
System administration
Data warehouse
Database administration
IBM DB2
New Relic
JD
Data
Streaming
Administration
Management
Virtual machines
Docker
Kubernetes
Design
Encryption
SSL
Authentication
LDAP
Authorization
Apache ZooKeeper
Metrics
Computer hardware
Storage
Research
Open source
Oracle
MySQL
JDBC
Change data capture
Unix
Scripting
Grafana
Web servers
Database
Linux
SLES
Red Hat Enterprise Linux
Shell
Red Hat Linux
OCP
VMware
Nexus
RBAC
Enterprise resource planning
Middleware
PostgreSQL
SaaS
Software deployment
Continuous integration
Continuous delivery
Jenkins
Terraform
Ansible
Cloud computing
Microsoft Azure
Amazon Web Services
Job Details
The JD is as follows: Role - Kafka Tech Admin
Job Responsibility
Mandatory Job Requirements
Manage single and multi-node Kafka cluster deployed on VM, Docker and Kubernetes Container platform.
Experience with Confluent Platform running on-prem
* Perform Kafka Cluster build, including Design, Infrastructure planning, High Availability and Disaster Recovery
* Implementing wire encryption using SSL, authentication using SASL/LDAP & authorization using Kafka ACLs in Zookeeper, Broker/Client, Connect cluster/connectors, Schema Registry, REST API, Producers/Consumers, Ksql
* Perform high-level, day-to-day administration and support functions
* Upgrades for the Kafka Cluster landscape comprising of Development, Test, Staging and Production/DR systems
* Creation of key performance metrics, measuring the utilization, performance, and overall health of the cluster.
* Capacity planning and implementation of new/upgraded hardware and software releases as well as for storage infrastructure.
* Research and recommend innovative ways to maintain the environment and where possible, automate key administration tasks.
* Ability to work with various infrastructure, administration, and development teams across business units
* Document and share design, build, upgrade and standard operating procedures. Conduct knowledge transfer sessions and workshops for other members in the team. Provide technical expertise and guidance to new and junior members in the team
* Create topics, setup Apache Kafka MirrorMaker 2, Confluent Replicator to replicate the topics, create connect clusters, Schemas for the topics using Confluent Schema Registry
* Configure various Opensource and licensed Kafka Source/Sink Connectors such as KafkaConnect for SAP HANA, Debezium Oracle and MySQL Connectors, Confluent JDBC source/sink, Confluent ADLS2 Sink connector and Confluent Oracle CDC source connector...
* Develop and maintain Unix scripts to perform day to day Kafka Admin and Security related functions using Confluent REST Proxy server
* Setting up monitoring tools such as Prometheus, Grafana to scrape metrics from various Kafkacluster components (Broker, Zookeeper, Connect, REST proxy, Mirror Maker, Schema Registry ...) and other endpoints such as webservers, databases, logs etc. and configure alerts for KafkaCluster and supporting infrastructure to measure availability and performance SLAs
* Experience with Confluent ksql to query and process Kafka streams
* Knowledge of Kafka Producer and Consumer APIs, Kafka Stream Processing, Confluent Ksql
* Availability to work in shifts, extended hours and to provide on-call support as required. There will be work over weekends at times depending on the project needs.
* Must have excellent communications and interpersonal skills
Preferred but Optional skills
* Linux (SLES or RHEL) system administration (basic or advanced), creating shell scripts ..
* Working experience on docker and Kubernetes clusters (opensource, Rancher, RedHat OCP, VMWare Tanzu) involving administration of containers (Operator level skills), deployments, updates, integration with products running outside of the cluster
* Working knowledge with container registry such as Harbor, Quay, Nexus etc. Exposure to Container/artifact scanners such as Trivy, Claire
* Security related config for above listed software or any other tools in SSL for wire encryption, integration with AD for authentication and RBAC for authorizations
* Implemented and supported any enterprise product such as any well-known ERP products, Data warehouse, Middleware etc.
* Database administration skills in Oracle, MSSQL, SAP HANA, DB2, Aerospike, Postgres ..
* Exposure to SaaS based observability platform like New Relic
* Deployment of container images and pods using CI/CD pipelines using Jenkins or comparable tools.
* Experience in building Kafka deployment pipelines using Terraform, Ansible, Cloud formation templates, shells etc. Worked in Public cloud environment such as Azure or AWS or Google Cloud Platform, preferably in Azure
Job Responsibility
- Primary responsibility of Confluent Platform Administrator will be to build, test and maintain Kafka Cluster and its eco-system which is deployed to run Data Streaming use cases for various business units
- Experience Overall 5+ years of experience out of which 2+ years around Confluent Platform administration
Mandatory Job Requirements
Manage single and multi-node Kafka cluster deployed on VM, Docker and Kubernetes Container platform.
Experience with Confluent Platform running on-prem
* Perform Kafka Cluster build, including Design, Infrastructure planning, High Availability and Disaster Recovery
* Implementing wire encryption using SSL, authentication using SASL/LDAP & authorization using Kafka ACLs in Zookeeper, Broker/Client, Connect cluster/connectors, Schema Registry, REST API, Producers/Consumers, Ksql
* Perform high-level, day-to-day administration and support functions
* Upgrades for the Kafka Cluster landscape comprising of Development, Test, Staging and Production/DR systems
* Creation of key performance metrics, measuring the utilization, performance, and overall health of the cluster.
* Capacity planning and implementation of new/upgraded hardware and software releases as well as for storage infrastructure.
* Research and recommend innovative ways to maintain the environment and where possible, automate key administration tasks.
* Ability to work with various infrastructure, administration, and development teams across business units
* Document and share design, build, upgrade and standard operating procedures. Conduct knowledge transfer sessions and workshops for other members in the team. Provide technical expertise and guidance to new and junior members in the team
* Create topics, setup Apache Kafka MirrorMaker 2, Confluent Replicator to replicate the topics, create connect clusters, Schemas for the topics using Confluent Schema Registry
* Configure various Opensource and licensed Kafka Source/Sink Connectors such as KafkaConnect for SAP HANA, Debezium Oracle and MySQL Connectors, Confluent JDBC source/sink, Confluent ADLS2 Sink connector and Confluent Oracle CDC source connector...
* Develop and maintain Unix scripts to perform day to day Kafka Admin and Security related functions using Confluent REST Proxy server
* Setting up monitoring tools such as Prometheus, Grafana to scrape metrics from various Kafkacluster components (Broker, Zookeeper, Connect, REST proxy, Mirror Maker, Schema Registry ...) and other endpoints such as webservers, databases, logs etc. and configure alerts for KafkaCluster and supporting infrastructure to measure availability and performance SLAs
* Experience with Confluent ksql to query and process Kafka streams
* Knowledge of Kafka Producer and Consumer APIs, Kafka Stream Processing, Confluent Ksql
* Availability to work in shifts, extended hours and to provide on-call support as required. There will be work over weekends at times depending on the project needs.
* Must have excellent communications and interpersonal skills
Preferred but Optional skills
* Linux (SLES or RHEL) system administration (basic or advanced), creating shell scripts ..
* Working experience on docker and Kubernetes clusters (opensource, Rancher, RedHat OCP, VMWare Tanzu) involving administration of containers (Operator level skills), deployments, updates, integration with products running outside of the cluster
* Working knowledge with container registry such as Harbor, Quay, Nexus etc. Exposure to Container/artifact scanners such as Trivy, Claire
* Security related config for above listed software or any other tools in SSL for wire encryption, integration with AD for authentication and RBAC for authorizations
* Implemented and supported any enterprise product such as any well-known ERP products, Data warehouse, Middleware etc.
* Database administration skills in Oracle, MSSQL, SAP HANA, DB2, Aerospike, Postgres ..
* Exposure to SaaS based observability platform like New Relic
* Deployment of container images and pods using CI/CD pipelines using Jenkins or comparable tools.
* Experience in building Kafka deployment pipelines using Terraform, Ansible, Cloud formation templates, shells etc. Worked in Public cloud environment such as Azure or AWS or Google Cloud Platform, preferably in Azure