Kafka Engineer / Administrator, Denver, Colorado (Onsite)
Key Responsibilities:
Design, deploy, configure, upgrade, and maintain highly available, scalable Apache
Kafka and Confluent Platform clusters for real-time data processing across production
and non-production environments.
Monitor platform health, broker performance, replication, partition balance, throughput,
storage utilization, consumer lag, and connector status; troubleshoot incidents and tune
the platform for reliability, performance, and minimal downtime.
Build and support Kafka topics, partitions, replication policies, schemas, service
accounts, and Kafka Connect integrations, including connectors for Amazon S3, Trino,
and OpenSearch; assist application and data teams with producer and consumer
integration.
Implement and maintain platform security and compliance controls, including TLS
encryption, SASL authentication, access control lists, role-based access control, secrets
management, auditability, and secure certificate lifecycle practices.
Automate cluster provisioning, configuration, scaling, upgrades, patching, and
operational tasks using Terraform, Ansible, CI/CD pipelines, and Python or shell
scripting.
Plan and test backup, recovery, failover, cross-cluster replication, and disaster recovery
procedures to support defined service-level objectives and business continuity
requirements.
Participate in production support and on-call rotation, lead technical troubleshooting,
document root cause analyses, and coordinate resolution with application teams,
infrastructure teams, incident managers, vendors, and cloud support personnel.
Provide technical guidance on Kafka architecture and operational best practices;
maintain runbooks, standards, diagrams, capacity plans, and support documentation;
and remain current with Apache Kafka and Confluent capabilities.
Required Qualifications:
3+ years of hands-on experience administering enterprise Apache Kafka or Confluent
Platform environments, including production cluster design, capacity planning,
upgrades, patching, performance tuning, and lifecycle management.
3+ years of experience troubleshooting complex Kafka issues involving broker
availability, replication, partitioning, rebalance behavior, consumer lag, throughput,
latency, storage, and Kafka Connect task failures.
3+ years of experience automating infrastructure and platform operations with Terraform
and Ansible, supported by Git-based CI/CD pipelines and Python or shell scripting.
Demonstrated experience designing and supporting highly available Kafka
architectures, including cross-cluster replication, failover, backup, recovery, and disaster
recovery testing.
Deep understanding of Kafka security and governance, including TLS, SASL, ACLs,
RBAC, Schema Registry compatibility, service accounts, credential management, and
audit controls.
Experience designing and operating observability capabilities using Prometheus,
Grafana, Confluent Control Center, OpenSearch, or equivalent solutions, including
actionable dashboards and alert thresholds.
Proven ability to define engineering standards, advise application teams on producer
and consumer patterns, review platform designs, and lead technical resolution across
multiple teams.
Proven excellence in client-facing communication, technical documentation, incident
leadership, root cause analysis, and consulting skills in English.
Ability to work in office, participate in an on-call rotation, and collaborate with senior
client resources and vendor support teams.
Preferred Qualifications:
Experience with Confluent Platform or Confluent Cloud components such as KRaft or
ZooKeeper, Schema Registry, Kafka Connect, Control Center, Cluster Linking, and
stream-processing capabilities.
Experience operating Kafka on AWS, Kubernetes, Red Hat OpenShift, or another cloud
or container platform; AWS Managed Streaming for Apache Kafka (Amazon MSK)
experience is beneficial.
Experience with GitLab, GitHub Actions, Jenkins, Argo CD, or comparable CI/CD and
GitOps tooling.
Experience integrating Kafka with Amazon S3, Trino, OpenSearch, relational databases,
IBM MQ, data warehouses, or other enterprise source and sink systems.
Familiarity with data governance, schema evolution, topic naming and retention
standards, capacity forecasting, service-level objectives, and platform cost optimization.
Relevant certification or formal training in Apache Kafka, Confluent, AWS, Kubernetes,
Terraform, Ansible, or related technologies.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
- Dice Id: RTL155990
- Position Id: e6b5bad11314ac413f50927c58730f23
- Posted 4 days ago
