Kafka Architect -Remote

Role : Kafka Architect -Remote

Duration : 6-12months

Job Description :

**** Webcam interview; *** Long term project *** Remote ***

Key Responsibilities

• Architect and implement real-time event streaming platforms for use cases such as real-time inventory tracking, fraud detection, personalized promotions, and dynamic pricing.
• Evaluate and select Kafka deployment models (self-managed, Confluent, AWS MSK, Azure Event Hubs, etc.) based on business and scalability needs.
• Design high-availability Kafka clusters to handle massive transaction volumes in eCommerce, POS systems, and digital customer engagement.
• Define best practices for topic management, schema evolution (Schema Registry), and partitioning strategies for large-scale retail data flows.
• Enable real-time customer data streaming from multiple sources (POS, eCommerce platforms, loyalty programs, IoT devices).
• Build Kafka-powered data pipelines to integrate with ERP, CRM, WMS, OMS, and personalization engines.
• Develop stream processing solutions using Kafka Streams and ksqlDB for dynamic promotions, order fulfillment, and fraud detection.
• Ensure seamless integration with retail analytics platforms like Snowflake, Databricks, and Google BigQuery.
• Optimize Kafka infrastructure to support seasonal peaks, high traffic surges (e.g., Black Friday, Cyber Monday), and multi-channel order management.
• Implement disaster recovery, failover strategies, and high-availability configurations for mission-critical retail systems.
• Define security policies for data protection, compliance (PCI-DSS), and customer privacy regulations (GDPR, CCPA).
• Establish monitoring, logging, and alerting using tools like Prometheus, Grafana, Splunk, and Open Telemetry.
• Act as the Kafka SME for retail transformation projects, educating engineering and business teams on event-driven capabilities.
• Collaborate with eCommerce, supply chain, marketing, and data science teams to drive real-time decision-making.
• Partner with cloud architects, DevOps, and cybersecurity teams to ensure a secure, scalable Kafka ecosystem.

Required Skills

• 14+ years of experience in designing and implementing event-driven architectures, preferably in retail, eCommerce, or omnichannel environments.
• Deep expertise in Apache Kafka (Kafka Streams, Kafka Connect, ksqlDB, Schema Registry).
• Strong knowledge of real-time retail analytics, demand forecasting, and digital customer engagement.
• Experience integrating Kafka with retail POS systems, inventory management, and order fulfillment platforms.
• Hands-on experience with microservices and event-driven patterns (CQRS, SAGA, Event Sourcing).
• Experience working with cloud-based Kafka implementations (AWS MSK, Azure Event Hubs, Google Cloud Platform Pub/Sub).
• Proficiency in real-time inventory tracking, recommendation engines, and fraud detection systems.
• Experience with containerization (Docker, Kubernetes) and CI/CD pipelines for Kafka deployments.
• Strong background in monitoring tools and performance tuning for Kafka clusters