Kafka SME Architect

Kafka SME Architect

Location: 5 days onsite in Houston, TX

Top Skills Required:

• Kafka SME with hands on experience managing integrations and interfaces with 3rd party systems. The important thing to understand here is that we want the individual to have the experience of integrating with Endur (so an understanding of its tables and some patterns used etc) and NOT functional understanding of Endur i.e. do design & configuration of functional items
• Oil & Gas experience

We are seeking a Kafka Subject Matter Expert (SME) with extensive hands-on experience in managing integrations and interfaces with third-party systems. The ideal candidate will have deep technical expertise in Apache Kafka and proven experience in integrating with Endur, focusing on understanding its data structures, tables, and integration patterns not on functional design or configuration.

Your Impact:

• Design, implement, and maintain Kafka-based integration solutions for real-time and batch data exchange.
• Develop and manage interfaces between Kafka and third-party systems, ensuring reliability, scalability, and security.
• Work closely with technical teams to integrate Kafka with Endur, leveraging knowledge of Endur tables and common integration patterns.
• Troubleshoot and optimize Kafka clusters, topics, and consumer/producer configurations.
• Ensure high availability and fault tolerance in Kafka environments.
• Collaborate with architecture and development teams to define best practices for data streaming and integration.
• Provide technical guidance and mentorship on Kafka-related projects.

Your Skills & Experience:

• Bachelor s or Master s degree in Computer Science, Information Technology, or related field (or equivalent experience).
• Strong hands-on experience with Apache Kafka, including cluster setup, configuration, and performance tuning.
• Proven track record of building and managing integrations with third-party systems using Kafka.
• Experience integrating with Endur, specifically understanding its database tables and integration patterns.
• Solid understanding of data streaming concepts, event-driven architecture, and messaging systems.
• Proficiency in Java, Python, or similar programming languages for Kafka client development.
• Familiarity with schema management (Avro, Protobuf), Kafka Connect, and Kafka Streams.
• Knowledge of monitoring and troubleshooting tools for Kafka (e.g., Confluent Control Center, Prometheus, Grafana).
• Strong problem-solving skills and ability to work in a fast-paced environment.
• Experience with cloud-based Kafka deployments (AWS MSK, Confluent Cloud, etc.).
• Knowledge of CI/CD pipelines and DevOps practices for Kafka deployments.
• Familiarity with energy trading systems and related integration patterns.