Kafka Lead/ Architect for 6-12+ months Contract; 100% Remote

Job Title : Lead Kafka Consultant

Location : 100% Remote

Duration : 6-12+ months Contract

Summary:

We are seeking a seasoned Kafka Integration Specialist to lead the design and implementation of our event-driven architecture. You will be responsible for building high-throughput, scalable, and resilient integrations using Apache Kafka and the Spring Boot ecosystem. This role requires a "builder-architect" mindset someone who can visualize the data flow across the enterprise and write the production-grade code to make it a reality.

Qualifications:
To perform the job successfully, an individual must be able to perform each essential function satisfactorily, with or without reasonable accommodation. The requirements listed below are representative of the experience, skill, and ability required. Reasonable accommodation may be made to enable individuals with disabilities to perform the essential functions.

Experience: 10-12+ years of total software development experience.

• Spring Ecosystem: 5+ years of hands-on experience with Java (11/17+) and Spring Boot, specifically using Spring-Kafka or Spring Cloud Stream in production environments.
• Kafka Expertise: Deep understanding of Apache Kafka internals (partitioning strategies, consumer groups, offsets, and log compaction) and experience with Confluent Schema Registry (Avro/Protobuf).
• Architecture Patterns: Proven experience implementing Event-Driven Architecture (EDA), including patterns like SAGA, CQRS, and the Transactional Outbox pattern.
• Database Integration: Strong proficiency in SQL/NoSQL databases and experience handling distributed transactions.
• Testing: Experience writing integration tests for Kafka using Testcontainers or Embedded Kafka.

Preferred (Nice-to-Haves)

• Stream Processing: Practical experience with Kafka Streams or KSQL for real-time data aggregation and transformation.
• Cloud & DevOps: Experience deploying Kafka microservices on Kubernetes (K8s), OpenShift, or public clouds (AWS MSK, Azure Event Hubs).
• Observability: Familiarity with distributed tracing tools like Zipkin, Jaeger, or OpenTelemetry.
• Certifications: Confluent Certified Developer or Spring Certified Professional.

Essential Job Functions:

• Architectural Design: Design end-to-end event-driven workflows, including schema registry management (Avro/JSON) and topic modeling.
• Development: Build robust microservices using Spring Cloud Stream and Spring Kafka.
• Performance Tuning: Optimize Kafka producers and consumers for throughput, latency, and reliability (tuning acks, batch.size, and consumer group behaviors).
• Observability: Implement distributed tracing (Sleuth/Micrometer) and monitoring for Kafka-based pipelines.
• Mentorship: Establish best practices for error handling (Dead Letter Topics), retry logic, and stateful processing.
• Core Tech: 5+ years of Java/Spring Boot development with deep expertise in the Spring Kafka library.
• Kafka Mastery: Proven experience with Kafka Clusters, KSQL, and Kafka Streams for real-time processing.
• Schema Governance: Experience with Confluent Schema Registry or similar tools to manage data evolution.
• Data Consistency: Deep understanding of "Exactly-once" processing semantics and the Transactional Outbox Pattern.
• Cloud/DevOps: Experience deploying Kafka-based apps on Kubernetes (AKS/EKS/GKE) or using managed services like Confluent Cloud

Our Company:

We are the preferred Oracle partner in North America with the experience and capabilities to comprehensively support Oracle products from implementation to support. We complement our Oracle expertise with additional competencies to support infrastructure, custom development, integrations, reporting, automation, and much more.

At Vigilant, our mission is to deliver impactful and successful outcomes to companies we serve. Our goal is to develop deep & lasting relationships with the clients we partner with by exceeding expectations, in our innovative solutions, the quality of our services, and the value we deliver.
Please visit our website URL: for more details pertaining to our company.

Please share your resume and contact details, if interested.

Thanks,

P Kumar