Sr Databricks Engineer (Hybrid)

Overview

Remote
Depends on Experience
Contract - W2
Contract - 12 Month(s)

Skills

databricks
SQL
python
AI/ML
MLOps

Job Details

Title: Sr. Databricks Engineer

Duration: 12 Months

Location: Hybrid - Local office 2 days/week

Top 3 Skills - Data Bricks Expert, SQL, Python
Soft Skills - Comfortable with being the go-to person, Ability to collaborate with multiple teams, Ability to see big picture What you will do Defining and implementing scalable and flexible data analytics architecture based on current and future business needs mainly utilizing Databricks platform
Implementing, maintaining and optimizing complex data pipelines to process large amounts of data
Integration, consolidation, cleansing and structuring of large data sets to enable business insights, reporting and AI/ML use cases
Delivering business insights by consolidating various data sources to understand business performance (e.g. sales data, customer satisfaction data, digital analytics data)
Sourcing, collecting and researching new internal and open-source external data sets
Defining and implementing data governance processes
Plan and prepare data engineering and analysis work between technical teams
Engage with business stakeholders to prepare and coordinate data sharing and data transfer activities
Support deployment, monitoring and maintenance of ML/AI models What you bring Multiple years of experience in related roles utilizing Databricks platform and related technologies for solving data engineering challenges
Good understanding of concepts and technologies underpinning modern data stack (storage, compute, governance, DataOps, MLOps, etc.) with focus on Databricks platform
Experience in data modelling and designing/implementing scalable and efficient ETL/ELT pipelines on Databricks platform
Strong experience in SQL and Python
Understanding of AI/ML, and MLOps / DataOps experience on Databricks platform
Experience in MLFlow or equivalent tools is highly desirable
Understanding and experience in solving data governance challenges utilizing Unity catalog (or related technology) in the context of Databricks platform
Experience with optimizing all aspects of Databricks workloads
Excellent problem-solving skills and analytical mind
Excellent verbal and written communication skills including the ability to explain technical issues, solutions and results to a non-technical audience
Strong work ethic and willingness to share your opinion and challenge status quo

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.