IT Resident Solutions Architect - Databricks
USA Experience: 15+ Years overall (5+ Years hands-on Databricks Platform experience required) Databricks Certification: Mandatory
Visa Type: Citizens /
Contact: /
Contact Name: Sri
Role
Resident Solutions Architect – Databricks
Location: 100% Remote – USA Experience: 15+ Years overall (5+ Years hands-on Databricks Platform experience required) Databricks Certification: Mandatory
Job Summary
We are seeking a hands-on, customer-facing Resident Solutions Architect with deep Databricks Platform expertise, combined with strong CI/CD and Terraform skills. This is an architect-level role for someone who can work directly with enterprise clients, design scalable Lakehouse and data platform solutions, and stay hands-on through build, deployment, and production support — including infrastructure-as-code delivery of the Databricks platform itself.
Key Responsibilities
· Serve as a resident, customer-facing Databricks Solutions Architect, providing architectural guidance, best practices, and hands-on delivery for enterprise Lakehouse implementations.
· Design and implement scalable Lakehouse architectures using PySpark, Spark SQL, and Delta Lake.
· Own Unity Catalog governance and security design, including data access controls, lineage, and cross-workspace governance patterns.
· Build and operate production data pipelines using Delta Live Tables (DLT) and Databricks Workflows.
· Administer Databricks Workspaces and Accounts, including provisioning, access management, and platform configuration.
· Provision and manage Databricks infrastructure using Terraform, including workspace, cluster, Unity Catalog, and job/workflow resources as reusable, version-controlled modules.
· Design and implement CI/CD pipelines for Databricks assets (notebooks, DLT pipelines, jobs, ML models) using GitHub Actions, GitLab CI, Jenkins, Azure DevOps, or equivalent, including Databricks Asset Bundles.
· Perform Spark performance tuning — partitioning, caching, Adaptive Query Execution (AQE), and data skew mitigation — to optimize cost and performance at scale.
· Design and manage SQL Warehouses for BI and analytics workloads.
· Build streaming data solutions using Kafka and Structured Streaming.
· Architect Databricks solutions across AWS, Azure, or Google Cloud Platform, tailored to each cloud's native services and security model.
· Partner directly with client engineers, business stakeholders, and executives to translate business problems into Databricks/Lakehouse solutions and measurable outcomes.
· Guide client teams through migration from legacy ETL platforms (e.g., Informatica) to Databricks.
Required Skills
· 15+ years of experience in Data Engineering, Cloud Engineering, Solutions Architecture, or Platform Engineering.
· Databricks Certification required (e.g., Databricks Certified Data Engineer Professional, Databricks Certified Solutions Architect, or equivalent).
· 5+ years of hands-on Databricks Platform experience, including:
· PySpark, Spark SQL, and Delta Lake
· Lakehouse Architecture design and implementation
· Unity Catalog governance and security
· Delta Live Tables (DLT)
· Databricks Workflows
· Workspace and Account Administration
· SQL Warehouses
· Strong hands-on Terraform experience with Databricks, including reusable modules for workspace, cluster, and Unity Catalog provisioning.
· Strong CI/CD experience (GitHub Actions, GitLab CI/CD, Jenkins, Azure DevOps, or equivalent), including CI/CD for Databricks Asset Bundles, notebooks, and jobs.
· Strong Spark performance tuning expertise — partitioning, caching, AQE, and data skew resolution.
· Kafka and Structured Streaming experience.
· Strong experience with AWS, Azure, or Google Cloud Platform.
· Proven customer-facing experience providing architectural guidance and Databricks best practices to enterprise clients.
· Strong communication and stakeholder-management skills, with the ability to translate business needs into practical Lakehouse solutions.
Preferred Skills
· MLflow for model tracking and lifecycle management.
· Databricks AI capabilities (AI/BI, Databricks Assistant, Mosaic AI).
· Experience migrating legacy ETL platforms (e.g., Informatica) to Databricks.
· Familiarity with the latest Databricks features — Genie, Lakebase, and Databricks Apps.
· Experience working with enterprise clients in a consulting or professional-services environment.
· Kubernetes, Docker, and cloud-native application architecture.
Contact: /
Never miss an opportunity! Create an alert based on the job you applied for.