Databricks Architect

Overview

Remote
$Market
Full Time
Accepts corp to corp applications
Contract - Independent
Contract - W2
Contract - 12+ month(s)

Skills

Migration
Databricks
Data Engineering
Cloud (AWS/Azure)
Lakehouse
Medallion
Data Mesh

Job Details

Job Title: Databricks Architect
Location: Remote
Duration: 12+ months

Job Description:

  • Architecting and Leadership:
    • Deep understanding of modern data architecture, including Lakehouse, Medallion architecture, and Data Mesh.
  • Data Engineering:
    • Strong programming experience in Python; Scala is a significant plus.
    • Proficiency in complex query tuning.
    • Experience building Slowly Changing Dimensions (SCD) TYPEs.
    • Familiarity with DBT.
    • Experience with structured streaming.
    • Knowledge of data formats like Iceberg, Parquet, and Delta.
    • Skills in Spark tuning and optimization.
    • Git and CI/CD integration
  • Migration Experience:
    • Demonstrated experience with large-scale migrations involving over 100 pipelines.
    • Proficiency with various migration tools.
    • Experience with other Enterprise Data Warehousing (EDW) platforms such as Snowflake and Redshift.
  • Familiarity with Databricks Technologies:
    • Unity Catalog
    • Delta Live Tables (DLT)
    • Databricks SQL (DBSQL)
    • Autoloader
    • Photon engine
    • Strong cultural fit within the Databricks ecosystem.
  • Cloud Technologies:
    • Experience with Azure.
    • Experience with AWS.
    • Terraform is a plus.
  • Consulting Experience:
    • Proven customer-facing experience.
  • Overall - Strong Databricks skills and robust data engineering experience
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.