Scala Spark Specialist

Job Title: Scala Spark Specialist

Lead strategic migration from Java to Scala Spark modules as a senior Big Data engineer. Individually design, develop, test, and productionize high-performance Scala Spark applications with Spring Boot backends and MSSQL integration.

Location: Toronto, ON
Experience: 8+ years Big Data engineering
Primary Focus: Java Scala Spark migration, hands-on individual contribution

Role Summary

Execute strategic code migration from existing Java-based Spark modules to optimized Scala Spark implementations. Work independently to design, develop, test, and deploy production-ready modules ensuring superior code quality, performance, and scalability. Integrate with Spring Boot REST APIs and Microsoft SQL Server.

Primary Responsibility

Java to Scala Spark Migration

Core Migration Mandate:

Refactor existing Java Spark modules Scala Spark (performance optimized)

Maintain 100% functional equivalence + performance improvements

Individual ownership: design develop test productionize

Zero supervision required for end-to-end delivery

Production-grade code quality and scalability

Hands-On Development

• Individually architect Scala Spark modules replacing Java implementations
• Develop, test, and productionize complete modules with full ownership
• Optimize performance achieving measurable improvements over Java baseline
• Ensure scalability for enterprise-grade data volumes and processing demands

Required Technical Expertise

Experience Profile (Must-Have)

8+ years Big Data engineering

Apache Spark + Hadoop ecosystem mastery (HDFS, Hive, YARN)

Expert-level Scala + Java (proven code migration experience)

Spring Boot REST API development

MSSQL database integration and optimization

Individual contributor: design develop deploy own

Nice-to-Have (Strong Advantage)

Cloud migration: On-premise Azure/AWS (Spark on cloud)

Machine learning concepts/workflows familiarity

Performance optimization case studies (before/after metrics)

Cloudera/Databricks managed Spark experience

Success Metrics

Migration velocity: X Java modules Scala per sprint

Performance uplift: Y% faster Spark execution

Code quality: Zero production defects post-migration

Scalability: Handle Zx original data volumes

Individual ownership: 100% end-to-end delivery

Keywords: Big Data Developer, Spark Scala Developer, Java Scala migration, Apache Spark, Hadoop HDFS Hive YARN, Scala programming, Java Spark migration, Spring Boot, RESTful APIs, Microsoft SQL Server MSSQL, code refactoring, performance optimization, scalability engineering, individual contributor, hands-on development, productionize, cloud migration Azure AWS, machine learning workflows, Toronto Big Data jobs, Spark DataFrames Datasets Streaming SQL, backend development, database optimization, Maven Gradle Git CI/CD, enterprise data engineering

About VDart Group
VDart Group is a global leader in technology, product, and talent solutions, serving Fortune 500 clients in 13 countries. With over 4,000 professionals worldwide, we deliver innovation, operational excellence, and measurable outcomes across industries. Guided by our commitment to People, Purpose, and Planet, VDart is recognized with an EcoVadis Bronze Medal and as a UN Global Compact member, reflecting our dedication to sustainable practices.