AWS Databricks Data Engineer

Los Angeles, CA, US • Posted 1 hour ago • Updated 11 minutes ago

Contract Corp To Corp

Contract Independent

Contract W2

On-site

Tror

Fitment

Dice Job Match Score™

🛠️ Calibrating flux capacitors...

Job Details

Skills

Data Analysis
Data Pipelines
Governance
Git
databricks
Data Warehousing
Data Governance
Business Intelligence
Performance Tuning
GitLab
Apache Spark
scalability
Cloud Computing
Continuous Integration
DevOps
Team Working
Workflows
Software Version Control
Problem Solving
Amazon Web Services
Identity and Access Management
Python (Programming Language)
Regulatory Compliance
Safety Principles
SQL Databases
Architecture
Extract Transform Load (ETL)
Information Engineering
Data Streaming
Amazon S3
Reliability
Dashboards
Access Controls
Data Systems
Data Lakes
Real Time Data
Spark Streaming
Pyspark
Non-relational Database
Query Optimisation
Security Policies

Summary

Job Title: AWS Databricks Data Engineer

Job Location: Los Angeles CA (Hybrid)

Hire type: FTE / CTH

Note: Only Locals to California

Job Description

We are seeking a highly skilled AWS Data Engineer with strong expertise in SQL, Python, PySpark, Data Warehousing, and Cloud-based ETL to join our data engineering team. The ideal candidate will design, implement, and optimize large-scale data pipelines, ensuring scalability, reliability, and high performance. This role requires close collaboration with cross-functional teams and business stakeholders to deliver modern, efficient data solutions.

Key Responsibilities

1. Data Pipeline Development

Build and maintain scalable ETL/ELT pipelines using Databricks on AWS.
Leverage PySpark/Spark and SQL to transform and process large, complex datasets.
Integrate data from multiple sources including S3, relational/non-relational databases, and AWS-native services.

2. Collaboration & Analysis

Partner with downstream teams to prepare data for dashboards, analytics, and BI tools.
Work closely with business stakeholders to understand requirements and deliver tailored, high quality data solutions.

3. Performance & Optimization

Optimize Databricks workloads for cost, performance, and efficient compute utilization.
Monitor and troubleshoot pipelines to ensure reliability, accuracy, and SLA adherence.
Apply query optimization, Spark tuning, and shuffle minimization best practices when handling tens of millions of rows.

4. Governance & Security

Implement and manage data governance, access control, and security policies using Unity Catalog.
Ensure compliance with organizational and regulatory data handling standards.

5. Deployment & DevOps

Use Databricks Asset Bundles for deployment of jobs, notebooks, and configuration across environments.
Maintain effective version control of Databricks artifacts using GitLab or similar tools.
Use CI/CD pipelines to support automated deployments and environment setups.

Technical Skills (Required)

Strong expertise in Databricks (Delta Lake, Unity Catalog, Lakehouse Architecture, Table Triggers, Workflows, Delta Live Pipelines, Databricks Runtime, etc.).
Proven ability to implement robust PySpark solutions.
Hands on experience with Databricks Workflows & orchestration.
Solid knowledge of Medallion Architecture (Bronze/Silver/Gold).
Significant experience designing or rebuilding batch heavy data pipelines.
Strong background in query optimization, performance tuning, and Spark shuffle optimization.
Ability to handle and process tens of millions of records efficiently.
Familiarity with Genie enablement concepts (understanding required; deep experience optional).
Experience with CI/CD, environment setup, and Git-based development workflows.
Solid understanding of AWS cloud, including:
IAM
Networking fundamentals
Storage integration (S3, Glue Catalog, etc.)

Preferred Experience

Experience with Databricks Runtime configurations and advanced features.
Knowledge of streaming frameworks such as Spark Structured Streaming.
Experience developing real-time or near real-time data solutions.
Exposure to GitLab pipelines or similar CI/CD systems.

Certifications (Optional)

Databricks Certified Data Engineer Associate / Professional
AWS Data Engineer or AWS Solutions Architect certification

Thanks & Regards

Akhil

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.

Dice Id: 91135853
Position Id: 2026-346
Posted 1 hour ago

Company Info

About Tror

TROR is an artificial intelligence consultancy specializing in developing powerful and customized Al solutions for business. With top Al Experts we take pride in providing the best cutting-edge Al consultancy. Our years of experience in various industries helps us to develop and implement bespoke Al solutions for businesses. Our on demand Al products have helped over 100 companies drive transformational results.

The solutions we bring on your table meet the highest industry standards and quality, effectively and efficiently resolving your issues and optimizing the way you want to move forward in the market. Through our customer centric approach, we ensure that we are always there for our valuable customers by offering them satisfactory solutions for guaranteed results.

Create job alert

Never miss an opportunity! Create an alert based on the job you applied for.