Title: Healthcare Databricks Claude AI Solutions Architect
Location: Remote,
Duration: 12+ Months
What you will do
Build Claude applications and agents
• Implement RAG, document intelligence, and multi-step agents with the Claude Developer Platform (Messages API, tool use, structured outputs, prompt caching, thinking, citations, Files API, batch processing), the Claude Agent SDK, and Agent Skills.
• Build and integrate MCP servers that expose Unity Catalog tables, Lakebase, and client APIs (FHIR, claims, CRM) as governed tools with least privilege.
• Call Claude natively on Databricks through Foundation Model APIs and AI Gateway (Anthropic Messages API and OpenAI-compatible endpoints), or through Amazon Bedrock, Google Cloud (Vertex AI), or Microsoft Foundry when the client requires it.
• Write prompts, tool schemas, and system instructions for healthcare tasks such as prior authorization review, clinical note summarization, claims and appeals triage, member and patient service, and medical coding support. Version them in the MLflow prompt registry.
• Use Claude Code for day-to-day engineering and codify repeatable workflows as Agent Skills.
Engineer the Databricks data and AI platform
• Build Lakeflow pipelines (Connect, Spark Declarative Pipelines, Jobs) in PySpark and SQL that land HL7 v2, FHIR R4, C-CDA, X12 EDI (837, 835, 270/271, 278), and EHR extracts into bronze, silver, and gold Delta tables.
• Build PHI de identification, tokenization, and masking steps. Tag and govern PHI with Unity Catalog row filters, column masks, ABAC, lineage, and audit.
• Build and tune Mosaic AI Vector Search indexes, chunking, and embedding pipelines. Deploy agents and models with Model Serving, Agent Framework, Agent Bricks, and Databricks Apps.
• Ship with CI/CD: Databricks Asset Bundles, GitHub Actions or Azure DevOps, unit and integration tests, and promotion across dev, test, and prod workspaces.
Quality, safety, and operations
• Build evaluation suites in MLflow 3 (tracing, LLM as judge scorers, golden datasets, regression tests). Gate releases on groundedness, PHI leakage, task success, latency, and cost per transaction.
• Implement guardrails, human-in-the-loop checkpoints, rate limits, retries, fallbacks, and caching. Monitor endpoints with AI Gateway usage tracking, inference tables, and system tables.
• Optimize cost: route across Opus, Sonnet, and Haiku by task, apply prompt caching and batch processing, and right-size serving endpoints, jobs, and serverless compute.
• Document designs and runbooks, take part in code and prompt reviews, and support production incidents on a rotation.
Must have
Claude and generative AI
• 4+ years building LLM applications in production, including 2 years of hands-on Claude development (Messages API, tool use, structured outputs, prompt caching, thinking, batch processing).
• Hands-on with the Claude Agent SDK and MCP: building or integrating MCP servers and tools, managing permissions and tool results.
• Practical RAG and agent evaluation experience: retrieval tuning, groundedness checks, test datasets, cost and latency measurement.
• Working knowledge of Claude on Databricks (Foundation Model APIs, AI Gateway) and at least one of Amazon Bedrock, Google Cloud (Vertex AI), or Microsoft Foundry.
Databricks (deep, hands-on)
• 4+ years of hands-on Databricks engineering at enterprise scale.
• Expert in PySpark, Spark SQL, and Python. Delta Lake internals: liquid clustering, OPTIMIZE and VACUUM, change data feed, time travel. Streaming with Structured Streaming or Spark Declarative Pipelines.
• Unity Catalog: catalogs, schemas, volumes, row filters, column masks, ABAC, lineage, system tables, Delta Sharing.
• Mosaic AI: Model Serving (Foundation Model APIs, external models, custom models), AI Gateway, Vector Search, Agent Framework, Agent Bricks, MLflow 3 tracing and evaluation, Databricks Apps, Lakebase.
• Databricks Asset Bundles, Databricks CLI and SDK, REST APIs, Git integration, secret scopes, and service principals. Cost tuning for jobs, serverless compute, and SQL warehouses.
Healthcare domain
• 5+ years working with healthcare data for payers, providers, PBMs, or life sciences.
• Hands-on with HL7 v2, FHIR R4, C-CDA, X12 EDI (837, 835, 270/271, 278), claims and encounter data, EHR extracts, and terminologies (ICD-10, CPT, HCPCS, SNOMED CT, LOINC, RxNorm, NDC).
• Working knowledge of HIPAA Privacy and Security Rules, HITECH, and HITRUST controls as they apply to code, prompts, logs, and pipelines, including de-identification (Safe Harbor, Expert Determination).
• At least one healthcare AI or analytics solution shipped to production.
Engineering fundamentals
• 10+ years of software or data engineering, with 3+ years at senior level.
• Strong SQL, Python, REST APIs, async patterns, and automated testing (pytest).
• Comfortable with US client stakeholders and an offshore team. Able to work US business hours. [Work authorization requirement, if any.]
Nice to have
• Claude for Healthcare connectors and Agent Skills (CMS Coverage Database, ICD-10, NPI Registry).
• Databricks healthcare accelerators: dbignite for FHIR, patient 360, claims analytics; Databricks Clean Rooms.
• Clinical NLP, medical coding automation, and vision models for scanned forms, faxes, and handwritten documents.
• Multi-agent systems with Supervisor Agent; LangGraph or OpenAI Agents SDK agents deployed on Databricks Apps.
• Epic (Clarity, Caboodle, FHIR APIs), Oracle Health, Availity, or clearinghouse integrations.
• Terraform for Databricks and cloud resources; Azure Databricks networking (VNet injection, Private Link) or the AWS equivalents.
• FastAPI, Streamlit, or Gradio front ends on Databricks Apps; Lakebase (Postgres) for agent state and memory.
• Life sciences: GxP and 21 CFR Part 11 validated environments.
Education and certifications
• Bachelor's degree in Computer Science, Engineering, or a related field.
• Databricks Certified Data Engineer Professional, held or obtained within 90 days of start.
• Databricks Certified Generative AI Engineer Associate strongly preferred; Machine Learning Associate a plus.
• Cloud certification a plus: Microsoft Certified Azure AI Engineer Associate or AWS Certified Data Engineer Associate.
• HL7 FHIR Proficiency a plus.
• Anthropic's Claude developer training a plus.
