Senior Kafka Administrator

Senior Kafka Administrator

Job Title: Senior Kafka Administrator
Location: Denver, CO or St. Louis, MO
Interview: Face-to-Face
 

Key Responsibilities

• Design, deploy, and operate highly available Kafka clusters across on-prem, cloud, and managed environments.
• Manage topics, partitions, quotas, retention policies, and consumer groups.
• Lead Kafka upgrades, patches, migrations, and cluster rebuilds.
• Administer Kafka Connect, Schema Registry, MirrorMaker/Confluent Replicator, and REST Proxy.
• Perform Kafka performance tuning, throughput testing, and benchmarking.
• Manage producer/consumer configurations including batching, compression, acknowledgments, ISR, and controller health.
• Perform capacity planning, partition strategy, and cluster right-sizing.
• Support AWS MSK environments as needed.
• Implement automation for provisioning, deployment, monitoring, and operational processes.

Contract Deliverables

• Develop a comprehensive hardware refresh plan, including capacity modeling, sizing, architecture diagrams, migration/cutover strategy, and risk assessment.
• Implement and validate refreshed on-prem Kafka clusters with performance benchmarks.
• Create operational standards, runbooks, monitoring/alerting configurations, backup/restore procedures, and DR playbooks.
• Conduct knowledge-transfer sessions and provide complete documentation handoff.

Required Qualifications

• 5+ years of experience in Systems/Platform Engineering, SRE, or DevOps.
• 4+ years of production Kafka administration at scale.
• Deep knowledge of Kafka internals, including partitions, replication, retention, compaction, and rebalancing.
• Hands-on experience with:
  • Apache Kafka / Confluent Platform
  • Kafka Connect
  • Schema Registry
  • MirrorMaker / Confluent Replicator
• Strong Linux fundamentals.
• Strong networking knowledge including TCP, DNS, and load balancing.
• Strong scripting/automation skills.
• Experience with Datadog, Grafana, JMX Exporter, and log aggregation.
• Experience with Disaster Recovery, multi-region architecture, and incident management.
• Strong technical documentation and communication skills.

Preferred Qualifications

• Experience with AWS MSK and Apache Kafka integration.
• Experience executing hardware refreshes, major Kafka cluster rebuilds, and migrations with minimal downtime.

Must Have: Kafka, Confluent Platform, Kafka Administration, Kafka Connect, Schema Registry, MirrorMaker/Replicator, Linux, Performance Tuning, Monitoring/Observability, Automation/Scripting.