Senior Data Quality/Governance Analyst (Remote)

Position Summary

Enterprise Tech Stack Storage and Processing for Governance, Data Management, Orchestration, Business Analytics, and Advanced Analytics to map to our cleint’s Strategic Objective by Implementing a statewide data governance framework that encourages data sharing among state agencies while adhering to privacy and security regulations. The contractor will architect and implement the Data Governance and Quality program from requirements through first working implementation, under strategic initiatives to enable a strong cross-agency data culture, unified digital wallet, and a data access request enterprise solution.

The role has two parts.

• Governance and Data Quality analysis: gathering requirements, defining the operating model and writing a procurement-ready SOW.
• Hands-on implementation:  
  • Writing data quality and conformed dimension analysis scripts for those assets. A conformed dimension is a shared reference entity, such as agency, location, person or date, that must mean the same thing in every source.
    • Declarative quality rules in pipelines and anomaly detection
    • Schema evolution handling
    • Quality scorecards and enforcement built into ETL/ELT
  • Publishing the scripts and their results in one version-controlled repository with a user-facing interface.

The work gives the client and the agencies a working governance foundation that the data mesh enforces and the data literacy program teaches.

Key Responsibilities 

Data Governance and Data Quality Requirements & SOW 

• Hold discovery sessions with the client, agency, business and data stakeholders to record current data ownership, quality issues and governance gaps.
• Write business, functional and non-functional requirements for the catalog, glossary, lineage, data quality and stewardship workflows.
• Draft the Governance SOW: scope, deliverables, acceptance criteria, milestones and assumptions for later implementation phases.
• Define the operating model, including the roles of data owners and stewards, a RACI and a decision-rights matrix.
• Build a requirements traceability matrix that links each requirement to strategic objectives and budgeted outcomes.

            Microsoft Purview Asset Setup 

• Register and scan priority data sources in Purview, including data lakes/lakehouses, transactional systems and other key agency data sources that may include DB2 and Microsoft Access.
• Set up the Purview collections structure and domains, and assign permissions in line with the operating model.
• Build and fill in the business glossary using systems that can be leveraged by all agencies. Link glossary terms to technical assets and assign owners and stewards.
• Configure classifications and sensitivity labels, including the PII classifications needed by the Digital Wallet and ID Gateway.
• Validate and add to lineage for priority data flows, working with the data mesh build.
• Set up Purview Data Quality rules and scoring for priority assets where that is available.

Data Quality & Conformed Dimension Analysis 

• Identify the conformed dimensions shared across sources and develop scripts for data quality checks.
• Write reusable SQL and Python/PySpark scripts that:  
• profile assets for completeness, validity, uniqueness, consistency and timeliness 
• check whether conformed dimensions match across sources, covering key matching, value-domain alignment, orphan records and hierarchy consistency 
• Parameterize the scripts so they can be rerun on new assets and new domains without rewriting them.
• Record the issues found and work with stewards on fixing them and turn the confirmed rules into data contracts.

Consolidated Repository & Interface 

• Set up a single version-controlled repository (Azure DevOps or GitHub) for all data quality scripts, conformed dimension definitions, rule metadata and documentation, with naming standards and contribution guidelines.
• Automate the scripts to run on a schedule through Databricks, Fabric or Azure Data Factory, and store the results in a central data quality results store.
• Deliver a user-facing interface over the repository and its results: a Power BI data quality dashboard that shows scores, trends and open issues by asset, domain and agency, with links back to the matching Purview assets and glossary terms.
• Document how stewards and agency teams add new rules and assets.

Required Qualifications 

• 5+ years of combined business analysis and hands-on data work on data governance, data quality or enterprise data programs.
• Microsoft Purview: source registration and scanning, collections, glossary, classifications and lineage. Experience/some exposure would be sufficient.
• Strong SQL and Python/PySpark skills for data profiling and quality analysis.
• Experience with dimensional modeling and analyzing conformed dimensions across sources.
• Experience with ADO or Git-based version control and scheduling scripts in Databricks, Fabric or Data Factory.
• Experience building dashboards (either Power BI or Tableu).
• Has written requirements packages and SOWs for technology procurement.
• Working knowledge of DAMA-DMBOK or a similar framework.