Lead Data Software Engineer with Databricks with Apache Kafka, Apache Spark, Kubernetes

Remote • Posted 1 hour ago • Updated 1 hour ago
Full Time
Remote
Fitment

Dice Job Match Score™

👾 Reticulating splines...

Job Details

Skills

  • Data Analysis
  • Microsoft Azure
  • Databricks
  • Apache Cassandra
  • Management
  • Testing
  • Unit Testing
  • Demonstrations
  • Product Demonstration
  • Mentorship
  • Apache Spark
  • Kubernetes
  • Python
  • Data Engineering
  • Cloud Computing
  • Migration
  • Team Leadership
  • Communication
  • English
  • Java
  • Apache Kafka
  • Streaming
  • Ansible
  • Continuous Delivery
  • Test-driven Development

Summary

We are looking for a Lead Data Software Engineer to drive the migration of an existing data analytics platform, originally built on Azure resources such as Data Factory, Databricks, Event Hub, and Cassandra, to a cloud-agnostic platform deployable on-premises. Responsibilities Implement streaming and batch Spark pipelines running on Kubernetes Lead the implementation of objects within the data generator framework Oversee deployment and testing across local and development environments Drive improvement of the current, actively changing solution Conduct bug investigation and resolution Perform unit testing to ensure solution quality Participate in refinement, planning, and demo sessions Guide and mentor team members throughout the implementation phase Requirements 5+ years of experience with Apache Spark, Confluent/Apache Kafka, and Kubernetes Proficiency in Python Familiarity with the data engineering domain and cloud-agnostic platform migrations Capability to dive deeper into new technologies in a short time during active implementation phases Ability to work independently with syncs with a team lead Strong communication skills and proactiveness Showcase of being a hands-on team player and technical leader Proficiency in English at a B2+ level Nice to have Knowledge of Java Familiarity with Kafka Connect, KSQL, and Kafka Streams Skills in Ansible and Argo CD Understanding of TDD
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
  • Dice Id: 10330481
  • Position Id: 7f650ccdc7939d610cc8585b152ebfe
  • Posted 1 hour ago
Create job alert
Set job alertNever miss an opportunity! Create an alert based on the job you applied for.

Similar Jobs

Remote

Today

Full-time

USD 134,470.00 - 203,400.00 per year

Remote

Today

Full-time

USD 190,000.00 - 240,000.00 per year

Remote

Today

Full-time

USD 142,000.00 - 196,600.00 per year

Remote

Today

Full-time

USD 148,750.00 - 201,250.00 per year

Search all similar jobs