DataBricks Architect

Overview

On Site
$60 - $65
Contract - Independent
Contract - W2

Skills

Data bricks
AWS
Azure
apache spark
delta lake
scala
python
SQL
terraform
medallion architecture

Job Details

Data bricks Architect
Oakland sfo, CA
12 plus years candidate
Only Local

We are seeking a highly skilled Databricks Architect to lead the design and implementation of scalable data solutions using the Databricks platform. This role will be responsible for creating robust data pipelines, optimizing Spark workloads, and integrating cloud-native architectures (Azure, AWS, or Google Cloud Platform) to support advanced analytics, data lakes, and real-time data processing.

Key Responsibilities

  • Design and architect end-to-end Databricks solutions for large-scale data platforms.

  • Build and optimize data pipelines, ETL/ELT workflows, and streaming jobs using Apache Spark.

  • Collaborate with data scientists, analysts, and engineers to integrate machine learning and AI pipelines into production workflows.

  • Develop and enforce data architecture standards, best practices, and governance policies.

  • Lead the migration of on-prem data workloads to cloud-native platforms using Databricks.

  • Support and mentor data engineers on Spark tuning, delta lake, and performance optimization.

  • Work with stakeholders to define technical requirements and translate business needs into architecture solutions.

  • Ensure security, compliance, and scalability of the data solutions.

Required Skills & Qualifications

  • 8+ years of experience in data engineering or architecture, with 2+ years hands-on with Databricks.

  • Strong expertise in Apache Spark, Delta Lake, and distributed data processing.

  • Proficiency in Python, SQL, and optionally Scala.

  • Experience with cloud platforms: Azure Databricks, AWS (with EMR or Databricks), or Google Cloud Platform.

  • Familiarity with CI/CD, DevOps, and infrastructure as code (Terraform/ARM).

  • Solid understanding of data modeling, data lakes, data warehouses, and modern data architectures (e.g., medallion architecture).

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.

About Intento Analytics LLC