Overview
On Site
$55 - $70
Contract - W2
Contract - 6 Month(s)
Skills
Scala
Scala on Apache Spark
Scala on Spark
Spark
Kubernetes
AKS
Azure
SQL
Azure Container Registry
Azure Data Lake Storage
Apache Airflow
Consolidated Audit Trail
Consolidated Audit Trail (CAT)
Job Details
Job Summary:
We are seeking a highly skilled and experienced Senior Data Engineer / Tech Lead with strong hands-on expertise in Scala on Spark, Kubernetes, and SQL, to join our growing data engineering team. The ideal candidate will be a seasoned developer capable of navigating and contributing to a complex, existing Scala-Spark codebase, while also delivering new features and enhancements in a fast-paced, highly collaborative environment.
Key Responsibilities:
- Take ownership of development and support of existing Scala on Spark pipelines deployed on Kubernetes.
- Analyze, troubleshoot, and resolve defects in a mature, distributed data processing environment.
- Implement enhancements and optimize performance of data applications.
- Collaborate effectively with team members across US, Canada, and India.
- Contribute in an individual contributor capacity while supporting team goals and project timelines.
- Demonstrate a strong sense of ownership and initiative in managing tasks.
- Present solutions and architectural ideas clearly in both internal and client-facing meetings.
Must-Have Qualifications:
- 8+ years of strong experience in Scala on Apache Spark, Kubernetes (preferably AKS), SQL (strong proficiency required)
- Proven ability to quickly understand and contribute to existing Scala codebases.
- Strong debugging and performance-tuning skills in distributed data environments.
- Excellent communication and collaboration skills.
Preferred Qualifications:
- Experience working with Microsoft Azure services: AKS (Azure Kubernetes Service), ACR (Azure Container Registry), ADLS (Azure Data Lake Storage)
- Experience with Apache Airflow for workflow orchestration.
- Familiarity with Financial Services regulations, especially Consolidated Audit Trail (CAT).
- Demonstrated technical thought leadership and client interaction experience.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.