Apply Now

Lead Data Software Engineer with Databricks with Apache Kafka, Apache Spark, Kubernetes

Remote • Posted 8 hours ago • Updated 8 hours ago

Full Time

Remote

Fitment

Dice Job Match Score™

To see how well you match this job, please log in or create an account.

Once logged in, be sure to complete your profile to get the most accurate match score.

🔢 Crunching numbers...

Job Details

Skills

Data Analysis
Microsoft Azure
Databricks
Apache Cassandra
Management
Testing
Unit Testing
Demonstrations
Product Demonstration
Mentorship
Apache Spark
Kubernetes
Python
Data Engineering
Cloud Computing
Migration
Team Leadership
Communication
English
Java
Apache Kafka
Streaming
Ansible
Continuous Delivery
Test-driven Development

Summary

We are looking for a Lead Data Software Engineer to drive the migration of an existing data analytics platform, originally built on Azure resources such as Data Factory, Databricks, Event Hub, and Cassandra, to a cloud-agnostic platform deployable on-premises. Responsibilities Implement streaming and batch Spark pipelines running on Kubernetes Lead the implementation of objects within the data generator framework Oversee deployment and testing across local and development environments Drive improvement of the current, actively changing solution Conduct bug investigation and resolution Perform unit testing to ensure solution quality Participate in refinement, planning, and demo sessions Guide and mentor team members throughout the implementation phase Requirements 5+ years of experience with Apache Spark, Confluent/Apache Kafka, and Kubernetes Proficiency in Python Familiarity with the data engineering domain and cloud-agnostic platform migrations Capability to dive deeper into new technologies in a short time during active implementation phases Ability to work independently with syncs with a team lead Strong communication skills and proactiveness Showcase of being a hands-on team player and technical leader Proficiency in English at a B2+ level Nice to have Knowledge of Java Familiarity with Kafka Connect, KSQL, and Kafka Streams Skills in Ansible and Argo CD Understanding of TDD

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.

Dice Id: 10330481
Position Id: 57cb44be43601dddd26c7a7be7b20fa9
Posted 8 hours ago

Create job alert

Never miss an opportunity! Create an alert based on the job you applied for.

Similar Jobs

Data Engineer with Databricks

Remote

•

19d ago

Job Title: Data Engineer with Databricks Location: Remote Job Type: Contract Job Summary: The Databricks Data Engineer owns the end-to-end migration strategy, target architecture design, and technical execution of moving legacy ETL workloads to the Databricks Lakehouse. He will establish migration standards, optimize PySpark pipelines, orchestrate complex data workflows, and deploy proprietary automation tools to ensure a seamless, high-performing transition from legacy systems. Key Responsibili

Easy Apply

Contract

65 - 70

Forward Deployed Engineer (Principal)

Remote or North Carolina

•

Today

Please Note: 1. If you are a first time user, please create your candidate login account before you apply for a job. (Click Sign In > Create Account) 2. If you already have a Candidate Account, please Sign-In before you apply. Job Description: Position Description We are seeking aForward Deployed Engineer (Principal) to serve as an elite technical authority, enterprise strategist, and field to engineering leader across the Infrastructure Software Division (ISG). In this high impact role, yo

Full-time

USD 110,800.00 - 177,300.00 per year

Forward Deployed Engineer (Principal)

Remote or California

•

Today

Full-time

USD 132,200.00 - 211,500.00 per year

Forward Deployed Engineer (Principal)

Remote or Colorado

•

Today

Full-time

USD 132,200.00 - 211,500.00 per year

Search all similar jobs

Remote jobs at EPAM Systems