IT Resident Solutions Architect - Databricks

IT Resident Solutions Architect - Databricks

USA Experience: 15+ Years overall (5+ Years hands-on Databricks Platform experience required) Databricks Certification: Mandatory

Visa Type: Citizens /  

 Contact:  /

Contact Name:  Sri

 Role

Resident Solutions Architect – Databricks

Location: 100% Remote – USA Experience: 15+ Years overall (5+ Years hands-on Databricks Platform experience required) Databricks Certification: Mandatory

Job Summary

We are seeking a hands-on, customer-facing Resident Solutions Architect with deep Databricks Platform expertise, combined with strong CI/CD and Terraform skills. This is an architect-level role for someone who can work directly with enterprise clients, design scalable Lakehouse and data platform solutions, and stay hands-on through build, deployment, and production support — including infrastructure-as-code delivery of the Databricks platform itself.

Key Responsibilities

·       Serve as a resident, customer-facing Databricks Solutions Architect, providing architectural guidance, best practices, and hands-on delivery for enterprise Lakehouse implementations.

·       Design and implement scalable Lakehouse architectures using PySpark, Spark SQL, and Delta Lake.

·       Own Unity Catalog governance and security design, including data access controls, lineage, and cross-workspace governance patterns.

·       Build and operate production data pipelines using Delta Live Tables (DLT) and Databricks Workflows.

·       Administer Databricks Workspaces and Accounts, including provisioning, access management, and platform configuration.

·       Provision and manage Databricks infrastructure using Terraform, including workspace, cluster, Unity Catalog, and job/workflow resources as reusable, version-controlled modules.

·       Design and implement CI/CD pipelines for Databricks assets (notebooks, DLT pipelines, jobs, ML models) using GitHub Actions, GitLab CI, Jenkins, Azure DevOps, or equivalent, including Databricks Asset Bundles.

·       Perform Spark performance tuning — partitioning, caching, Adaptive Query Execution (AQE), and data skew mitigation — to optimize cost and performance at scale.

·       Design and manage SQL Warehouses for BI and analytics workloads.

·       Build streaming data solutions using Kafka and Structured Streaming.

·       Architect Databricks solutions across AWS, Azure, or Google Cloud Platform, tailored to each cloud's native services and security model.

·       Partner directly with client engineers, business stakeholders, and executives to translate business problems into Databricks/Lakehouse solutions and measurable outcomes.

·       Guide client teams through migration from legacy ETL platforms (e.g., Informatica) to Databricks.

Required Skills

·       15+ years of experience in Data Engineering, Cloud Engineering, Solutions Architecture, or Platform Engineering.

·       Databricks Certification required (e.g., Databricks Certified Data Engineer Professional, Databricks Certified Solutions Architect, or equivalent).

·       5+ years of hands-on Databricks Platform experience, including:

·       PySpark, Spark SQL, and Delta Lake

·       Lakehouse Architecture design and implementation

·       Unity Catalog governance and security

·       Delta Live Tables (DLT)

·       Databricks Workflows

·       Workspace and Account Administration

·       SQL Warehouses

·       Strong hands-on Terraform experience with Databricks, including reusable modules for workspace, cluster, and Unity Catalog provisioning.

·       Strong CI/CD experience (GitHub Actions, GitLab CI/CD, Jenkins, Azure DevOps, or equivalent), including CI/CD for Databricks Asset Bundles, notebooks, and jobs.

·       Strong Spark performance tuning expertise — partitioning, caching, AQE, and data skew resolution.

·       Kafka and Structured Streaming experience.

·       Strong experience with AWS, Azure, or Google Cloud Platform.

·       Proven customer-facing experience providing architectural guidance and Databricks best practices to enterprise clients.

·       Strong communication and stakeholder-management skills, with the ability to translate business needs into practical Lakehouse solutions.

Preferred Skills

·       MLflow for model tracking and lifecycle management.

·       Databricks AI capabilities (AI/BI, Databricks Assistant, Mosaic AI).

·       Experience migrating legacy ETL platforms (e.g., Informatica) to Databricks.

·       Familiarity with the latest Databricks features — Genie, Lakebase, and Databricks Apps.

·       Experience working with enterprise clients in a consulting or professional-services environment.

·       Kubernetes, Docker, and cloud-native application architecture.

Contact:  /