Overview
Remote
Depends on Experience
Contract - W2
Contract - 6 Month(s)
Skills
Scala
SQL
Azure Synapse
Spark
GitHub
Spark UI
Java
Azure Repos
Maven
Job Details
Title: Lead Scala Development Consultant
Location: Remote (Must be based in US)
Duration: 6 months
MOI: Phone & Video
Description:
- Lead data discovery and analysis efforts to establish data contracts from various source systems.
- Develop and enhance data pipelines using Scala, SQL, and Azure Synapse Pipelines.
- Monitor, support, and maintain data pipelines for ingestion and transformation using Scala and SQL on Spark.
- Troubleshoot and resolve pipeline errors and performance issues.
- Identify and address data discrepancies and inconsistencies.
- Translate technical details into understandable insights for non-technical stakeholders.
- Use GitHub for source and version control of code/configuration artifacts.
- Deploy code using GitHub Workflows/Actions.
Requirements:
- 7+ years of experience with Spark in data engineering roles.
- Strong understanding of distributed computing and Spark UI for troubleshooting.
- Proficient in Scala and Java.
- Familiarity with Zone-based/Medallion architecture and Delta/Parquet file formats.
- Experience with Azure Synapse.
- Hands-on experience with GitHub, Azure Repos, and build tools like Maven.
- Strong analytical, problem-solving, and communication skills.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.