Sr. Kafka Engineer

Sr. Kafka Engineer

Location: Irvine, CA (Hybrid)

Duration: Long Term Contract

Required Experience: 12+ years

Job Summary

We are looking for a Senior Kafka Engineer with strong experience in Kafka, AWS, Java, and distributed systems. The candidate will be responsible for designing, implementing, supporting, and optimizing Kafka solutions in a large-scale enterprise environment.

The role requires strong troubleshooting, automation, and platform engineering skills, along with the ability to work closely with application, infrastructure, DevOps, and architecture teams.

Key Responsibilities

·        Design, implement, and support Kafka clusters, topics, producers, and consumers.

·        Manage Kafka partitions, replication, retention, performance, and capacity.

·        Integrate Kafka with Java/Spring Boot and Python applications.

·        Troubleshoot Kafka and production issues and perform root-cause analysis.

·        Implement Kafka security, including SSL, SASL, and ACLs.

·        Support Kafka in AWS environments and support Kafka deployments in                                       Docker/Kubernetes environments.

·        Monitor Kafka using tools such as Splunk and Datadog.

·        Develop automation using Python, Ansible, and Terraform.

·        Improve platform reliability, scalability, and operational efficiency through SRE and                     DevOps practices.

·        Identify repetitive operational tasks and develop automation and self-service solutions.

·        Work with development and architecture teams to design reliable real-time data solutions.

·        Participate in incident response, problem management, technology evaluations, and                   platform improvements.

Required Skills & Experience

·        10+ years of experience in software, platform, or infrastructure engineering.

·        5+ years of hands-on Kafka experience in production environments.

·        Strong understanding of Kafka architecture, brokers, topics, partitions, replication,                       consumer groups, and offsets.

·        Strong AWS experience, preferably AWS MSK.

·        Strong Java development experience, preferably Spring Boot and microservices.

·        Experience integrating Kafka with Java and Python applications.

·        Experience with Kafka Connect and Schema Registry.

·        Experience with Kafka Streams or ksqlDB is a plus.

·        Experience with Terraform, Ansible, Docker, and Kubernetes/OpenShift.

·        Experience with monitoring and observability tools such as Splunk, PrometheGrafana,           or Datadog.

·        Understanding of Kafka security and access controls.

·        Strong troubleshooting, automation, and root-cause analysis skills.

·        Good understanding of SRE, DevOps, CI/CD, Git, and Jenkins.

·        Excellent communication and collaboration skills.

Preferred Skills

·        Confluent Kafka / Confluent Cloud experience.

·        Experience building self-service platforms and automation.

·        Experience with distributed tracing, APM, and synthetic monitoring.