Balyasny Asset Management
Balyasny Asset Management

Senior Software Engineer - Kafka Platform

• Posted 60+ days ago • Updated 4 days ago
Full Time
Fitment

Dice Job Match Score™

🛠️ Calibrating flux capacitors...

Job Details

Skills

  • Software Development
  • Identity Management
  • Lifecycle Management
  • Workflow
  • Capacity Management
  • Provisioning
  • Budget
  • Incident Management
  • Dashboard
  • Network
  • Documentation
  • Mentorship
  • Communication
  • Partnership
  • Collaboration
  • ISR
  • Storage
  • Replication
  • Recovery
  • Software Engineering
  • Terraform
  • Continuous Integration
  • Continuous Delivery
  • GitHub
  • Jenkins
  • Python
  • Java
  • Bash
  • Linux
  • File Systems
  • Reliability Engineering
  • Modeling
  • Performance Tuning
  • TLS
  • OAuth
  • ACL
  • RBAC
  • Management
  • Auditing
  • Regulatory Compliance
  • CruiseControl
  • Microsoft Azure
  • Google Cloud Platform
  • Google Cloud
  • Computer Networking
  • Cloud Computing
  • Amazon Web Services
  • Roadmaps
  • Data Processing
  • Apache Kafka
  • Apache Flink
  • Apache Spark
  • Streaming
  • Semantics
  • Kubernetes
  • Change Data Capture
  • Database
  • Disaster Recovery

Summary

We are seeking a software engineer with strong Kafka experience to help build and evolve BAM's event-streaming platform. This role is ideal for an engineer who combines hands-on software development skills with deep knowledge of Kafka and distributed systems.

You will design, build, and operate the core services, tooling, and automation that power our Kafka platform. In addition to maintaining a reliable, secure, and scalable streaming environment, you will write production-quality code to improve self-service adoption, developer experience, and platform resilience across the firm

What you'll do
  • Develop internal software, APIs, tooling, and automation to simplify Kafka provisioning, access management, topic lifecycle management, and operational workflows.
  • Architect, deploy, and operate production-grade Kafka clusters (self-managed and/or Confluent/MSK), including upgrades, capacity planning, multi-AZ/region DR, and performance tuning.
  • Operate Kafka on Kubernetes using Operators, Helm, and GitOps, and build IaC-driven automation with guardrails for repeatable, compliant, zero-downtime provisioning and deployments.
  • Implement and manage Kafka Connect, Schema Registry, and MirrorMaker 2/Cluster Linking; standardize connectors (e.g., Debezium) and build self-service patterns.
  • Drive reliability: define SLOs/error budgets, on-call rotations, incident response, postmortems, runbooks, and automated remediation.
  • Implement observability: metrics, logs, traces, lag monitoring, and capacity dashboards (e.g., PrometheGrafana, Burrow, Cruise Control, OpenTelemetry).
  • Secure the platform: TLS/mTLS, SASL (OAuth/SCRAM), RBAC/ACLs, secrets management, network policies, audit, and compliance automation.
  • Guide event-streaming best practices: topic design, partitioning, compaction/retention, idempotency, ordering, schema evolution/compatibility, DLQs, EOS semantics.
  • Partner with app, data, and SRE teams; provide enablement, documentation, and internal tooling for a great developer experience.
  • Lead/mentor engineers and contribute to roadmap, standards, and platform strategy.

Required qualifications
  • Excellent communication and partnership skills with platform and application teams.
  • Deep hands-on experience operating Kafka in production at scale (brokers, controllers, partitions, ISR, tiered storage/retention, rebalancing, replication, recovery).
  • Strong software engineering fundamentals, with experience building and maintaining production systems.
  • Strong Kubernetes expertise running stateful systems.
  • Automation first: Infrastructure as Code (Terraform), Helm, Operators, GitOps (Argo CD/Flux), and CI/CD (e.g., GitHub Actions/Jenkins) for platform lifecycle.
  • Proficiency with one or more languages for tooling/automation: Python, Go, or Java; plus Bash and solid Linux fundamentals (networking, filesystems, JVM tuning basics).
  • Observability and reliability engineering for Kafka: PrometheGrafana, logging, alerting, lag monitoring, capacity/throughput modeling, performance tuning.
  • Security for data in motion: TLS/mTLS, SASL/OAuth, ACL/RBAC, secrets management (e.g., Vault), and audit/compliance practices.
  • Experience with Kafka ecosystem components: Kafka Connect, Schema Registry, MirrorMaker 2/Cluster Linking; familiarity with Cruise Control.
  • Cloud experience (AWS/Azure/Google Cloud Platform) with networking, IAM, and one or more managed offerings (e.g., Confluent Cloud or AWS MSK).
  • Proven track record designing runbooks, leading incidents/postmortems, and driving platform roadmaps.

Nice to have
  • Experience building internal developer platforms or self-service infrastructure products.
  • Data processing frameworks (Kafka Streams, Flink, Spark Structured Streaming) and EOS semantics.
  • Experience with Strimzi or Confluent for Kubernetes in production.
  • Knowledge of CDC patterns and tools (e.g., Debezium) and database connectors at scale.
  • Multi-region architectures, cluster linking strategies, and disaster recovery drills.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
  • Dice Id: 10125634
  • Position Id: 6e82cfa782984f0f1fada790e46c572b
  • Posted 30+ days ago
Create job alert
Set job alertNever miss an opportunity! Create an alert based on the job you applied for.

Similar Jobs

Lead DevOps Engineer - Kafka, Kubernetes, AWS

Hybrid in Chicago, Illinois

•

3d ago

Easy Apply

Full-time

$180,000 - $190,000

Lead DevOps Software Engineer

Hybrid in Chicago, Illinois

•

3d ago

Easy Apply

Full-time

Depends on Experience

Lead DevOps Engineer

Hybrid in Chicago, Illinois

•

4d ago

Easy Apply

Full-time

140,000 - 150,000

Lead AWS Cloud Engineering w/ Kubernetes

Hybrid in Chicago, Illinois

•

3d ago

Easy Apply

Full-time

$175,000 - $190,000

Search all similar jobs