Overview
On Site
Hybrid3 days onsite
Depends on Experience
Contract - Independent
Contract - W2
Skills
Databricks
PySparkv Scala
Bigdata
AWS
Airflow
CI/CD
Job Details
Data Engineer
Key Skills: Databricks, PySpark Scala, Bigdata.
Location : North Quincy MA - Hybrid Onsite
Skill Sets Required
- Must have over 10 years of experience in Data engineering.
- Good decision-making and problem solving skills
- Solid understanding of Databricks fundamentals/architecture and have hands on experience in setting up Databricks cluster, working in Databricks modules (Data Engineering and SQL warehouse).
- Solid Knowledge on medallion architecture, DLT and unity catalog within Databricks.
- Experience in migrating data from on-prem Hadoop to Databricks/AWS
- Understanding of core AWS services, uses, and AWS architecture best practices
- Hands-on experience in different domains, like database architecture, business intelligence, machine learning, advanced analytics, big data, etc.
- Solid knowledge on Airflow
- Solid knowledge on CI/CD pipeline in AWS technologies
- Application migration of RDBMS, java/python applications, model code, Opensearch etc.
- Solid programming background on scala and/or python with Spark
- Experience with Docker and Kubernetes is a plus
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.