Scala Big Data Engineer

mysql, postgres, big data systems development, java, spark, scala, ETL, ETL Pipelines, Data Lake, Real-time, batch ingestion, linux
Contract W2, 12 Months
$0 - $80
Travel not required

Job Description

Scala BigData Systems Dev
Sunnyvale, San Jose, San Francisco CA

12 Months plus possible extension or perm

We are looking for an expert level software engineer in Scala, Spark, Java and ML

You will be building one of the largest Big Data Platforms processing Petabytes of data.

The ETL Pipelines will deliver value to Business Analytics, Operational Analytics, Text Analytics and Data Services

This team has been tasked with a large scale migration of legacy applications and data feeds to a new unified data lake to enable cost savings and AI transformation.


Key Responsibilities:

  • Real-time/Batch Ingestion and Processing pipelines built using Apache Spark / Storm.
  • Machine Learning & Offstage analytics pipelines built using Apache Spark
  • Data platform & Search Systems built using HBase, Solr and ElasticSearch.
  • Present solutions to teams and stakeholders and document design decisions
  • Interface with DBAs, Security, Storage, and Networking to meet requirements
  • Support and Troubleshoot as required during UAT and Production
  • Participate in Performance Engineering activities for optimizations

Must have:

  • 7-8 years Industry experience of large scale bigdata systems development
  • Solid knowledge of Scala or Java with Spark including modern language features
  • Advanced skills in Databases like MySQL , Postgres, HBase, Cassandra
  • Strong knowledge of OO design paradigms.
  • Development experience in Linux/Unix environment.
  • B.S. in Computer Science or a related technical field.

Good to have:

  • Experience developing apps at scale on AWS or Azure
  • Exposure to AI/ML stacks like MXNet, PyTorch , Keras, Tensor

No Sub-Contract, Third Parties or Corp to Corp arrangements! W2 only!
Note: U.S. Citizens and those authorized to work in the U.S. without sponsorship will be considered. We are unable to sponsor at this time.

Send resume to

Follow us on LinkedIn for job updates.


Dice Id : 10454717
Position Id : 6097721
Originally Posted : 1 year ago
Have a Job? Post it