Lead Data Engineer Real-Time Streaming (Kafka/Flink)

Overview

Hybrid
Depends on Experience
Full Time

Skills

Amazon Web Services
Apache Flink
Apache Kafka
Caching
Change Data Capture
Cloud Computing
Communication
Continuous Delivery
Continuous Integration
Data Governance
DevSecOps
Docker
ELT
Extract
Transform
Load
Good Clinical Practice
Google Cloud Platform
Java
Jenkins
Kubernetes
Leadership
Machine Learning (ML)
Mentorship
Meta-data Management
Microservices
Microsoft Azure
Microsoft Windows
Migration
Python
Real-time
Redis
Regulatory Compliance
Roadmaps
SQL
Software Engineering
Streaming
Terraform

Job Details

Summary:
We re seeking a Lead Data Engineer to architect and scale real-time streaming platforms. You ll design event-driven microservices, optimize performance, and mentor engineers while working with cloud-native technologies.

Responsibilities:

  • Build high-throughput, low-latency streaming solutions using Kafka, Flink, Hazelcast.
  • Design ETL/ELT pipelines for real-time ingestion and enrichment.
  • Implement microservices (Java/Python) and event-driven architectures.
  • Optimize performance with distributed caching (Redis, Hazelcast).
  • Drive CI/CD, DevSecOps, and observability best practices.
  • Lead data governance for metadata, lineage, and compliance.
  • Mentor engineers and guide platform roadmap.

Required Skills:

  • 10+ years software engineering; 5+ years real-time/event-driven systems.
  • Expert in Kafka (brokers, partitions, Schema Registry), Flink (stateful ops, checkpointing).
  • Strong in ETL/ELT, streaming joins/windows, idempotent processing.
  • Cloud experience (AWS/Azure/Google Cloud Platform), Kubernetes, Docker.
  • CI/CD tools (Jenkins, Terraform), observability (Prometheus, ELK).
  • Excellent communication and leadership skills.

Nice to Have:

  • CDC, custom Kafka connectors, Flink SQL.
  • Streaming ML integration, multi-region DR.
  • Zero-downtime migrations, blue/green deployments.

About Capgemini:
Capgemini is an AI-powered global business and technology transformation partner, delivering tangible business value. We imagine the future of organizations and make it real with AI, technology and people. With our strong heritage of nearly 60 years, we are a responsible and diverse group of 420,000 team members in more than 50 countries. We deliver end-to-end services and solutions with our deep industry expertise and strong partner ecosystem, leveraging our capabilities across strategy, technology, design, engineering and business operations. The Group reported 2024 global revenues of 22.1 billion.
Make it real | ;/p>

Life at Capgemini:
Capgemini supports all aspects of your well-being throughout the changing stages of your life and career. For eligible employees, we offer:
Flexible work
Healthcare including dental, vision, mental health, and well-being programs
Financial well-being programs such as 401(k) and Employee Share Ownership Plan
Paid time off and paid holidays
Paid parental leave
Family building benefits like adoption assistance, surrogacy, and cryopreservation
Social well-being benefits like subsidized back-up child/elder care and tutoring
Mentoring, coaching and learning programs
Employee Resource Groups
Disaster Relief

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.

About Capgemini America, Inc.