Job Title: Kafka Engineer
Job Location: Downtown, New York/ Woodlawn, MD
Job Description:
Design, develop, and maintain robust Kafka-based applications and data pipelines that support business operations, including the real-time or near-real-time data to AI/ML models.
Collaborate with development, operations, and infrastructure teams to deliver reliable, scalable, and high-performing Kafka solutions.
Ensure the availability, reliability, and performance of Kafka clusters and related systems.
Work closely with architects, data engineers, and stakeholders to define requirements and deliver solutions.
Troubleshoot and resolve issues in Kafka applications, ensuring minimal downtime and optimal performance.
Document code, design decisions, processes, configurations, and best practices for future reference and team knowledge sharing.
Mentor junior developers and share Kafka expertise, fostering a culture of learning and growth.
Stay current with the latest Kafka releases, features, and ecosystem advancements.
Perform statistical analysis to monitor team performance, improve processes, and ensure customer satisfaction.
Define and set SLAs for projects, ensuring high standards of service delivery.
Applicants must qualify for the series and grade of the posted position. Experience must be IT related
Minimum Qualifications:
Grade 15 To qualify at the GS-15 level, you must have at least 52 weeks of specialized experience at the GS-14 level, or equivalent, leading the design, development, and implementation of enterprise-scale, fault-tolerant data pipelines using Apache Kafka; providing expert-level management and administration of Kafka clusters throughout the Systems Development Life Cycle (SDLC), including upgrades and patching; overseeing large-scale, cross-functional projects as a senior Product Owner or Agile/Scrum leader, ensuring alignment with organizational goals; demonstrating advanced proficiency in
Java programming, with experience in Python as a plus; architecting event-driven and microservices-based solutions leveraging Kafka APIs (Producer, Consumer, Streams, Connect); establishing and enforcing best practices for serialization formats (Avro, Protobuf, JSON) and schema registry/data governance, including Hackolade for data modeling; directing the optimization of producer/consumer performance and large-scale data ingestion strategies; leading the implementation of unit and integration testing frameworks for Kafka applications; managing hybrid integration architecture patterns and ensuring reliability, scalability, and performance of Kafka clusters; overseeing monitoring and troubleshooting activities using tools such as Prometheus and Grafana; providing technical guidance and mentorship to teams on Kafka cluster setup, configuration, and tuning; ensuring compliance with organizational standards and data governance policies.
Never miss an opportunity! Create an alert based on the job you applied for.
