Scala/Spark Developer

Overview

On Site
Depends on Experience
Full Time

Skills

Spark
ETL
Apache Spark
Scala
Apache Hadoop

Job Details

Job Title: Scala/Spark Developer
Location: Vancouver, Canada
Employment Type: Contract
Experience Level: Mid to Senior

About the Role

We are seeking a highly skilled Scala/Spark Developer to join our data engineering team in Vancouver. You will be responsible for developing, optimizing, and maintaining large-scale data processing applications. The ideal candidate has a strong background in distributed computing, big data technologies, and functional programming with a passion for solving complex data challenges.

Key Responsibilities

  • Design, develop, and maintain data pipelines using Apache Spark (batch and streaming) and Scala.
  • Implement scalable, high-performance ETL solutions for large datasets.
  • Optimize Spark jobs for speed, efficiency, and cost-effectiveness.
  • Work with Hadoop ecosystem components (HDFS, Hive, HBase, etc.).
  • Collaborate with data scientists, analysts, and other engineers to deliver business-critical data solutions.
  • Integrate data from multiple structured and unstructured sources.
  • Ensure code quality, testing, and documentation for all development work.
  • Participate in Agile/Scrum ceremonies and contribute to sprint planning and estimations.

Required Skills & Qualifications

  • Bachelor s or Master s degree in Computer Science, Engineering, or related field.
  • 5+ years of professional experience in Scala development.
  • 4+ years of experience with Apache Spark (Core, SQL, Streaming).
  • Strong understanding of functional programming concepts.
  • Hands-on experience with distributed data systems and big data frameworks.
  • Proficiency in working with HDFS, Hive, Parquet, and ORC file formats.
  • Solid experience with ETL pipelines and data transformation logic.
  • Familiarity with cloud platforms (AWS, Azure, or Google Cloud Platform) for big data workloads.
  • Strong SQL skills and experience with relational and NoSQL databases.

Preferred Qualifications

  • Experience with Kafka or other messaging/streaming platforms.
  • Exposure to Databricks or EMR for Spark job orchestration.
  • Understanding of CI/CD pipelines and DevOps practices.
  • Familiarity with machine learning workflows in a Spark environment.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.