Streaming Platform Engineer

Genesis10 is currently seeking a Streaming Platform Engineer for a contract to hire position based in Plano, TX.

Description:
We are seeking a Streaming Platform Engineer to design, implement, and operate high-throughput, low-latency event streaming systems using modern distributed messaging platforms (e.g., Apache Kafka, Apache Pulsar, Azure Event Hubs, Amazon Kinesis, Google Pub/Sub). You will build resilient producers/consumers, manage schema evolution, ensure exactly-once delivery, and integrate streaming data with RDBMS, NoSQL, and analytics systems.

Responsibilities:

Streaming Platform Architecture & Operations

• Design and manage production-grade streaming clusters (Kafka, Pulsar, Event Hubs, Kinesis, etc.) across cloud environments.
• Configure topics/namespaces/streams, partitioning strategies, retention, replication, and geo-redundancy.
• Implement schema management using Schema Registry, Pulsar Schema, Protobuf/Avro/JSON Schema, or cloud-native schema services.
• Ensure high availability, disaster recovery, and multi-region failover.

Producer & Consumer Development

• Write scalable, fault-tolerant producers and consumers in Java, Scala, Python, or Go.
• Build real-time data pipelines using:

• Kafka Streams / ksqlDB
• Pulsar Functions
• Flink
• Spark Structured Streaming
• Kinesis Data Analytics

• Guarantee exactly-once or at-least-once semantics, idempotency, and ordered processing.

Change Data Capture (CDC) & Integration

• Deploy CDC pipelines using Debezium, Maxwell, MongoDB Change Streams, or cloud-native CDC (DMS, Dataflow).
• Use connectors (Kafka Connect, Pulsar IO, Event Hubs Capture) to sync data to:

• Data warehouses (Snowflake, BigQuery, Redshift)
• Operational databases (AzureSQL, PostgreSQL, Cassandra, DynamoDB)

Database Integration & Optimization

• Design schemas, indexes, and partitioning in RDBMS (AzureSQL, PostgreSQL, MySQL) and NoSQL (Cassandra, DynamoDB, MongoDB) for high-velocity writes.
• Optimize query performance for event-sourced or streaming-derived data.
• Manage data consistency between streams and persistent stores (event sourcing, CQRS patterns).

Observability, Reliability & Automation

• Instrument end-to-end monitoring (Prometheus, Grafana, Datadog, CloudWatch, Azure Monitor).
• Set up consumer lag alerts, schema compatibility checks, and error dead-letter queues.
• Automate deployments with Terraform, Helm, Kubernetes Operators (Strimzi, Pulsar Operator), or cloud CLIs.
• Write contract tests, integration tests, and chaos engineering scenarios.

Requirements:

• Streaming Platforms: Apache Kafka or Apache Pulsar (deep expertise in at least one); familiarity with Kinesis, Event Hubs, Pub/Sub, Redpanda
• Programming: Go, Java, or Scala
• Processing Frameworks : Kafka Streams, ksqlDB, Pulsar Functions, Flink, Spark Streaming
• Schema & Serialization: Avro, Protobuf, JSON Schema, Schema Registry (Confluent, Apicurio, Pulsar Schema)
• CDC & Connectors: Debezium, Kafka Connect, Pulsar IO, AWS DMS, Azure CDC
• Databases: AzureSQL/PostgreSQL/MySQL (indexing, partitioning), Cassandra/DynamoDB/MongoDB
• Cloud & Infra: AWS, Google Cloud Platform, Azure; Docker, Kubernetes, Terraform, CI/CD

Preferred Skills:

• Migrated from Kafka to Pulsar (or vice versa) in production.
• Built multi-tenant streaming platforms with isolation and quota enforcement.
• Used event sourcing, CQRS, or domain-driven design with streams.
• Contributed to Strimzi, Pulsar Operators, or open-source connectors.
• Certified: Confluent Certified Developer, Databricks Apache Spark, AWS Data Analytics, etc.

Salary range from $125K to 150K; benefits package will be shared through the interview process.

W2 Status:
Only candidates available and ready to work directly as Genesis10 employees will
be considered for this position.

If you have the described qualifications and are interested in this exciting opportunity, please apply!

Genesis10 - Contract & Perm/Direct Hire
Ranked a Top Staffing Firm in the U.S. by Staffing Industry Analysts for six consecutive years, Genesis10 puts thousands of consultants and employees to work across the United States every year in contract, contract-for-hire, and permanent placement roles. With more than 300 active clients, Genesis10 provides access to many of the Fortune 100 firms and a variety of mid-market organizations across the full spectrum of industry verticals.

For contract roles, Genesis10 offers the benefits listed below. If this is a perm-placement opportunity, our recruiter can talk you through the unique benefits offered for that particular client. Benefits of Working with Genesis10:

• Access to hundreds of clients, most who have been working with Genesis10 for 5-20+ years.
• The opportunity to have a career-home in Genesis10; many of our consultants have been working exclusively with Genesis10 for years.
• Access to an experienced, caring recruiting team (more than 7 years of experience, on average.)
• Behavioral Health Platform
• Medical, Dental, Vision
• Health Savings Account
• Voluntary Hospital Indemnity (Critical Illness & Accident)
• Voluntary Term Life Insurance
• 401K
• Sick Pay (for applicable states/municipalities)
• Commuter Benefits (Dallas, NYC, SF)
• Remote opportunities available

For multiple years running, Genesis10 has been recognized as a Top Staffing Firm in the U.S., as a Best Company for Work-Life Balance, as a Best Company for Career Growth, for Diversity, and for Leadership, amongst others. To learn more and to view all our available career opportunities, please visit us at our website.

Genesis10 is an Equal Opportunity Employer. Candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or status as a protected veteran.
#DIG10-DA