Motion Recruitment Partners, LLC
Motion Recruitment Partners, LLC

Spark Framework Engineer - Enterprise Data Platform

CHARLOTTE, NC, US • Posted 30+ days ago • Updated 3 hours ago
Contract W2
On-site
$69.5 - $76.16/hr
Fitment

Dice Job Match Score™

👾 Reticulating splines...

Job Details

Skills

  • Financial Services
  • Ansible
  • Load Balancing
  • Data Processing
  • FOCUS
  • Use Cases
  • Onboarding
  • Usability
  • Productivity
  • Unity
  • Collaboration
  • IT Management
  • Design Review
  • Scala
  • Java
  • Python
  • API
  • Artificial Intelligence
  • Machine Learning (ML)
  • Apache Spark
  • SQL
  • Data Governance
  • Extract
  • Transform
  • Load

Summary

Outstanding contract opportunity! A well-known Financial Services Company is looking for an Ansible Automation Engineer (with Load Balancing experience) in Columbus or Charlotte (hybrid 3 days on-site). If you're passionate about advancing automation and working with cutting-edge load balancing technology within a collaborative, non-egotistical team environment, this is for you!

Contract Duration: 12 Months

We are seeking a Lead Software Engineer to drive the evolution of SparkFlow, an enterprise data processing framework built on Apache Spark. This role will focus on designing clean, composable abstractions, delivering new functional capabilities within the framework, and significantly improving developer experience. The engineer will also design and deploy AI enabled capabilities and lead the integration of the SparkFlow framework into the Unity control plane.

Key Responsibilities
  • Lead the architectural evolution of the SparkFlow framework, emphasizing modularity, composability, and clear abstraction boundaries
  • Design and implement new functional features within the framework to expand supported use cases and capabilities
  • Balance feature delivery with framework quality, ensuring performance, backward compatibility, and governance standards
  • Improve developer experience by simplifying APIs, configuration models, extension points, and onboarding patterns
  • Design and implement AI driven solutions that assist developers and enhance framework usability, productivity, and operability
  • Integrate SparkFlow as a first class framework within the Unity control plane
  • Collaborate with data, platform, and infrastructure teams to ensure consistent adoption across environments
  • Provide technical leadership through design reviews, code reviews, and architectural guidance
Required Experience
  • Deep experience with Apache Spark (Scala and/or Java; Python a plus)
  • Proven success building reusable frameworks or libraries, including ownership of functional feature development
  • Strong skills in API design, abstraction design, and extensibility patterns
  • Experience applying AI/ML or LLM based techniques to developer tooling or data frameworks
  • Experience working in large?scale, enterprise data environments
Nice to Have
  • Experience integrating frameworks into control planes or internal developer platforms
  • Familiarity with Spark SQL-centric architectures and data governance patterns
  • Experience modernizing or replacing legacy ETL frameworks
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
  • Dice Id: 10105282
  • Position Id: 867653
  • Posted 30+ days ago
Create job alert
Set job alertNever miss an opportunity! Create an alert based on the job you applied for.

Similar Jobs

Software Engineer

Charlotte, North Carolina

•

Today

Contract

USD 68.00 - 73.00 per hour

Software Engineer

Charlotte, North Carolina

•

Today

Contract

USD 69.00 - 74.00 per hour

Big Data Engineer PySpark, Scala - Hybrid 68683

Charlotte, North Carolina

•

Today

Easy Apply

Contract

$55.00 - 75.00

Software Engineer

Charlotte, North Carolina

•

Today

Contract

USD 69.00 - 74.00 per hour

Search all similar jobs