Location: Bay Area, CA (Onsite work)
Job description:
- Own data quality alerting and issue management in Databricks: alert coverage, triage, routing, SLAs, root-cause analysis and closure; keep DQ and abnormality checks current as pipelines change; report DQ health to stakeholders.
- Own metric definitions, calculations and measurement standards across functions; set certification standards and the governance gate metrics must pass before official reporting; mentor analysts, data engineers and BI developers.
- Work with analytics engineers to architect and govern the Databricks Gold layer: certified data models with embedded DQ and abnormality checks, metric views and data lineage.
- Drive data readiness for AI: tiered assessments scoring datasets on completeness, accuracy and automation readiness, and turn certified metrics into natural-language queryable experiences.
- Must-have skills
a. Databricks hands-on, including curated or Gold-layer data models [Databricks, Delta Lake, lakehouse, medallion, Gold layer].
b. Data quality ownership: DQ checks, anomaly or abnormality detection, alerting, SLAs, root-cause analysis [data quality, DQ, data observability, anomaly detection].
c. Metric governance: metric definitions and certification, semantic layer or metric views, data lineage, data governance [metric definitions, semantic layer, metric views, data lineage, data catalog].
d. Solutioning and transformation: has architected or led a data platform build, migration or modernization, not only operated pipelines [architect, solution design, transformation, migration, modernization].
- Nice-to-haves: supply chain or hardware operations data; AI readiness or natural-language query on metrics (text-to-SQL, AI/BI); mentoring or leading engineers; senior level by years and scope; profile signals open to work.
