Databricks Engineer

Overview

On Site
$40 - $50
Contract - W2
Contract - 12 Month(s)

Skills

Databricks
Hadoop
Python
Spark
Spark SQL
PySpark
AirFlow
IBM StreamSet
AWS

Job Details

Role - Databricks Engineer

Location: Boston, MA

Duration: 12 months
 

Databricks Engineer: 12+ years of total experience with 8 years relevant experience in the mandatory skills.

Mandatory Skills: Databricks, Hadoop, Python, Spark, Spark SQL, PySpark, AirFlow and IBM StreamSet

Required Skills & Experience:

Develop Data Engineering and ML pipelines in Databricks and different AWS services, including S3, EC2, API, RDS, Kinesis/Kafka and Lambda to build serverless applications

Solid understanding of Databricks fundamentals/architecture and have hands on experience in setting up Databricks cluster, working in Databricks modules (Data Engineering, ML and SQL warehouse).

Knowledge on medallion architecture, DLT and unity catalog within Databricks.

Experience in migrating data from on-prem Hadoop to Databricks/AWS

Understanding of core AWS services, uses, and AWS architecture best practices

Hands-on experience in different domains, like database architecture, business intelligence, machine learning, advanced analytics, big data, etc.

Solid knowledge on Airflow

Solid knowledge on CI/CD pipeline in AWS technologies

Application migration of RDBMS, java/python applications, model code, elastic etc.

Solid programming background on scala, python, SQL

 
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.