Technical Lead Databricks

Overview

Remote

Hybrid

Depends on Experience

Contract - W2

Contract - Independent

Contract - 12 Month(s)

Skills

Require databricks

AWS

Python

PySpark

Delta Lake

Lambda

Glue

EMR

EC2

IAM

Kinesis

CloudWatch

Job Details

Technical Lead Databricks(10-12 years)

Skills: Life science experince , Require databricks, AWS hands on skilled resources who can work on project , Python as well to build and develop model, agents.

Job Description

We are looking for an experienced Technical Lead Databricks with strong hands-on expertise in Databricks, AWS, Python, and Life Sciences domain. The ideal candidate will lead data engineering and advanced analytics initiatives, guide the team in solution design, and work closely with stakeholders to deliver scalable, high-quality data solutions.

Key Responsibilities

Lead end-to-end design, development, and implementation of Databricks-based data pipelines.
Architect and optimize ETL/ELT workflows using PySpark, Delta Lake, and Databricks SQL.
Work hands-on in AWS cloud services (S3, Lambda, Glue, EMR, IAM, CloudWatch, etc.).
Build, train, and deploy Python-based models, ML pipelines, and LLM/AI agents as required by project needs.
Collaborate with cross-functional teams in the Life Sciences domain to translate business requirements into technical solutions.
Ensure adherence to coding standards, best practices, performance optimization, and scalability.
Provide technical leadership, mentoring, and guidance to junior team members.
Drive technical discussions, design reviews, and solution validations.
Proactively identify risks, troubleshoot issues, and ensure timely delivery.

Required Technical Skills

10 12 years of experience in Data Engineering / Big Data / Cloud engineering.
Strong hands-on experience with Databricks (PySpark, Delta Lake, Notebooks, Clusters, Jobs).
Expertise in Python for building data pipelines, ML models, and automation scripts.
Solid experience with AWS cloud services:
- S3, Lambda, Glue, EMR, EC2, IAM, Kinesis, CloudWatch, Step Functions.
Experience working in Life Sciences / Pharma / Healthcare domain (mandatory).
Understanding of data governance, data quality, and regulatory requirements in Life Sciences.
Experience with MLflow, Feature Store, or model deployment frameworks is a plus.
Knowledge of AI/ML model building or LLM/agent development is an added advantage.
Strong problem-solving skills and ability to work in an agile, dynamic environment.

Soft Skills

Strong communication and stakeholder management skills.
Ability to lead technical teams and drive solutions independently.
Excellent analytical and decision-making capabilities.
Ability to work collaboratively with geographically distributed teams.

Bachelor s or Master s degree in Computer Science, Engineering, Data Science, or related field.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.