Databricks Architect

Hello;

Information Management / Databricks Architect

Long Term

Job Location- Remote

Operational Data Stack

SQL Server / T-SQL , COBOL Flat Files, Dapper / ORM, C# / .NET 8 ,ASP.NET Core APIs

Cloud & Analytics Platform

AWS (S3, Glue, Athena, RDS),Databricks / Apache Spark , Amazon Bedrock , Delta Lake ,AWS IAM / Lake Formation

Data Engineering & Governance

ETL / ELT Pipeline Design, Medallion Architecture , Data Catalog & Lineage, Data Governance Frameworks , BI / Semantic Layers

This role bridges TAI's deep SQL Server heritage with modern cloud data platforms (AWS, Databricks/Spark), enabling the analytics and AI capabilities that will differentiate TAI's next platform in market.

Core Responsibilities:

Design and govern TAI's enterprise data architecture - spanning the operational SQL Server core, cloud-native data stores on AWS, and modern analytics platforms including Databricks

Define the data platform strategy that supports TAI's evolution from client-installed deployments to a centralized, multi-tenant SaaS model - including data isolation, partitioning, and residency patterns for regulated insurance data

Architect data ingestion pipelines that consolidate data from TAI's diverse client environments (SQL Server, COBOL flat files, network-share repositories) into unified analytics-ready stores

Own data quality frameworks: define validation rules, lineage tracking, and observability tooling for all data pipelines

Expert knowledge of schema design, medallion architecture (Bronze/Silver/Gold), and data contract standards for central data lakes and analytical platform

Professional experience building AI data platforms - ensuring training datasets, feature stores, and inference pipelines are governed, reproducible, and audit-ready

Define and enforce data governance standards across TAI's platform - including data classification, PII handling, retention policies, and access controls appropriate for insurance-regulated environments

Ensure data architecture designs comply with state insurance regulatory requirements, SOC 2 controls, and client contractual obligations around data residency and sovereignty

Architect role-based data access patterns that support multi-client isolation: no client should be able to access or infer data belonging to another carrier

Own the data catalog and metadata management strategy - ensuring all data assets are discoverable, documented, and trusted

Partner with the TAI Architecture and Engineering to ensure data architecture aligns with platform architecture - no silos between operational systems and analytics

Mentor engineers in data engineering best practices: pipeline design, SQL optimization, dimensional modeling.

Engage with client-facing teams to understand reporting and data export requirements across TAI's client base, translating them into scalable platform features