Location: San Francisco, CA
Work Arrangement: Remote, PST Hours
Employment Type: Contract
Duration: 12 months
Position Overview
We are seeking a Databricks Resident Solution Architect with extensive experience in data engineering, data platforms, analytics, and modern data warehouse solutions. This client-facing role will lead Databricks implementations, provide architecture guidance, and work directly with clients to design and deliver scalable cloud-native data platforms and Databricks solutions.
Key Responsibilities
- Lead end-to-end Databricks implementations as a hands-on developer, technical lead, or solution architect.
- Serve as a client-facing technical advisor, understanding business requirements and recommending appropriate Databricks solutions.
- Design and implement cloud-native data platforms across AWS, Azure, and/or GCP.
- Architect scalable ETL/ELT pipelines using Databricks, Delta Lake, Structured Streaming, and modern data engineering frameworks.
- Design and optimize large-scale distributed data processing workloads for performance, scalability, and cost efficiency.
- Apply advanced Apache Spark expertise, including partitioning, execution plans, memory management, and performance optimization.
- Implement DevOps and CI/CD practices using Azure DevOps, GitHub Actions, GitLab CI/CD, Jenkins, or similar tools.
- Apply MLOps principles for machine learning lifecycle management, model deployment, and monitoring.
- Leverage Databricks Lakehouse capabilities including Delta Lake, Unity Catalog, Workflows, MLflow, and Delta Live Tables.
- Troubleshoot complex data platform challenges and recommend architecture patterns aligned with business and technical requirements.
- Support client discussions and demonstrate the value and capabilities of Databricks solutions.
Required Qualifications
- 12–15+ years of experience in data engineering, data platforms, data analytics, and modern data warehouse solutions.
- 10+ years of consulting and client-facing delivery experience.
- Demonstrated experience delivering 6–8+ end-to-end Databricks implementations.
- Advanced Databricks certification, such as Databricks Data Engineering Professional certification, with completion of relevant learning paths and coursework.
- Strong expertise designing and implementing cloud-native data platforms across AWS, Azure, and/or GCP, with deep hands-on expertise in at least one cloud platform.
- Advanced knowledge of Apache Spark, including performance optimization, partitioning strategies, execution plans, memory management, and runtime internals.
- Extensive hands-on experience with Databricks, Delta Lake, Structured Streaming, and scalable ETL/ELT pipelines.
- Experience implementing production-grade DevOps and CI/CD for data solutions.
- Working knowledge of MLOps, machine learning lifecycle management, model deployment, and monitoring.
- Strong knowledge of the Databricks Lakehouse Platform and its core capabilities.
- Experience tuning large-scale distributed workloads and designing highly performant, scalable, and cost-efficient solutions.
- Strong troubleshooting and solution architecture skills.
Preferred Qualifications
- Databricks Solutions Architect Champion program participation or equivalent advanced Databricks expertise.
- Experience working with public sector organizations
All qualified applicants will receive consideration for employment without regard to race, color, religion, sex (including pregnancy, sexual orientation, and gender identity), national origin, age, disability, genetic information, protected veteran status, or any other characteristic protected by applicable federal, state, or local law.
We are committed to providing reasonable accommodations to qualified applicants with disabilities and for sincerely held religious beliefs, as required by applicable law. Applicants who require a reasonable accommodation during the hiring process may request assistance.
Employment may be contingent upon the successful completion of job-related pre-employment screening, which may include background and reference checks and, where permitted by applicable law and applicable to the position, drug testing. All such screening will be conducted in accordance with applicable federal, state, and local laws.
Never miss an opportunity! Create an alert based on the job you applied for.