Software Engineer 3 - Contingent

Job Description:

• Senior Software Engineer - SparkFlow Framework (Contract)

• Client is seeking a Senior Software Engineer to contribute to SparkFlow, an enterprise data processing framework built on Apache Spark.
• This engineer will implement new functional framework features, strengthen existing components, improve developer ergonomics, and help deliver AI enabled capabilities that make the framework easier to use and operate.
• The role will also support integrating SparkFlow into the Unity control plane by building and hardening the required interfaces and workflows.

• Build and enhance new functional features in the SparkFlow framework (sources/targets, transformations, governance/controls, reliability features).
• Implement and refine framework extension points (APIs, configs, libraries) to improve composability and reuse.
• Improve developer experience: simplify configuration patterns (e.g., pipeline JSON/configs), reduce onboarding friction, and improve diagnostics/observability hooks.
• Develop AI enabled solutions that assist developers (e.g., guided config generation, validation, troubleshooting accelerators) and improve framework usability.
• Contribute to Unity control plane integration work: implement adapters/operators, automation, and integration testing for consistent orchestration.
• Participate in code reviews, design discussions, and on call/operational support patterns as needed.

• Strong hands on engineering experience with Apache Spark (Scala and/or Java; Python a plus), including Spark SQL-based processing.
• Experience building frameworks/libraries (not just applications), including API and abstraction design.
• Working knowledge of CI/CD and engineering fundamentals (Git, build tooling, unit/integration testing).
• Experience with enterprise data ecosystem components (e.g., Hadoop/Hive, Kafka, cloud storage/warehouse patterns) and production hardening.

• Experience improving pipeline onboarding and deployment patterns (config driven artifacts, launcher scripts, scheduler integration).
• Familiarity with governance capabilities like audit trail capture, metadata/lineage integration, and data in motion controls.
• Cloud/hybrid experience (e.g., Google Cloud Platform Dataproc patterns) supporting Spark workloads.

• Java or Scala, Spark, SQL, Shell scripting

• Consult on or participate in moderately complex initiatives and deliverables within Software Engineering and contribute to large-scale planning related to Software Engineering deliverables.
• Review and analyze moderately complex Software Engineering challenges that require an in-depth evaluation of variable factors.
• Contribute to the resolution of moderately complex issues and consult with others to meet Software Engineering deliverables while leveraging solid understanding of the function, policies, procedures, and compliance requirements.
• Collaborate with client personnel in Software Engineering.

• 4 plus years of Software Engineering experience, or equivalent demonstrated through one or a combination of the following: work or consulting experience, training, military experience, education.

• Experience of Java/Spark or Scala/Spark
• Experience using version control tools like GIT, CI/CD process and build tools like Gradle and Maven.
• Hands-on experience working with Hadoop Ecosystem and Big data technologies and stores like Hive, Kafka, S3 and Iceberg (good to have)
• Strong knowledge of Database concepts and UNIX bash scripting.
• Experience with Spark and SQL and Spark performance tuning knowledge, good to have - Data modeling, normalization
• Good to have experience on developing cloud native applications and cloud deployment ( AWS or Google Cloud Platform )

Pay Range: $XXX - $XXX

The specific compensation for this position will be determined by a number of factors, including the scope, complexity and location of the role as well as the cost of labor in the market; the skills, education, training, credentials and experience of the candidate; and other conditions of employment. Our full-time consultants have access to benefits including medical, dental, vision and 401K contributions as well as any other PTO, sick leave, and other benefits mandated by appliable state or localities where you reside or work.