Kafka Administrator

Location: St. Louis, MO 63131

Work Model: Onsite

Contract: 12 Months

Experience: 8–10+ Years

Interview: Face-to-Face Mandatory

Job Summary

We are seeking an experienced Kafka Administrator / Big Data Administrator to manage, support, automate, and optimize enterprise-scale Kafka, Hadoop, AWS, and DevOps platforms. The candidate will be responsible for platform availability, performance, security, scalability, automation, and production support.

Must-Have Skills

• Strong Apache Kafka Administration

• KRaft and AWS MSK

• AWS: EC2, S3, IAM, VPC, EBS, CloudWatch, CloudTrail

• Linux (RHEL/Rocky Linux)

• Shell/Bash scripting and Python

• Jenkins, Git, Ansible, Terraform

• Docker and Kubernetes

• Grafana, Prometheus, Splunk

• Kafka security: SSL, SASL, ACLs, encryption

• Kafka Connect, Schema Registry, Cruise Control, MirrorMaker

• Performance tuning, capacity planning, high availability

• Networking and security concepts

Key Responsibilities

• Deploy, configure, manage, monitor, and troubleshoot Kafka clusters and AWS MSK.

• Manage brokers, partitions, replication, consumer lag, cluster scaling, and capacity planning.

• Support Kafka security, performance tuning, and high-availability requirements.

• Manage AWS cloud infrastructure and support MSK, EMR, Lambda, Glue, and Airflow environments.

• Provision and automate infrastructure using Terraform, CloudFormation, and Ansible.

• Design and maintain CI/CD pipelines using Jenkins, GitHub Actions, or GitLab CI.

• Implement monitoring and alerting using Prometheus, Grafana, Splunk, ELK, or CloudWatch.

• Develop automation scripts using Python, Shell, or Bash.

• Support Docker/Kubernetes environments and enterprise production workloads.

• Provide L2/L3 production support, incident management, RCA, and preventive solutions.

• Maintain operational documentation and ensure security, audit, and compliance requirements.

Preferred Skills

• Cloudera CDP

• AWS MSK

• Airflow

• Spark

• Experience with large-scale/petabyte-scale data environments

• AWS, Google Cloud Platform, Kafka, or Kubernetes certifications