Role: Data Architect
Location: Cary, NC
Work Model: Day 1 onsite; hybrid schedule, Tuesday–Thursday onsite
Experience: 10–15 years
2. Mandatory Skills – Must-Have
- Python, PySpark, Scala and SQL – strong production-level data engineering skills.
- Azure Databricks – Delta Lake, Spark performance tuning, Jobs/Workflows and Unity Catalog.
- Azure data services – ADLS Gen2 and Azure Data Factory (ADF).
- Lakehouse / Medallion Architecture – proven experience implementing Bronze, Silver and Gold layers at enterprise scale.
- Data Pipeline Engineering – metadata-driven ingestion frameworks, data transformation, data quality and large-scale pipeline optimization.
- Production AI/LLM Experience – hands-on experience designing and deploying LLM-based solutions, including RAG, embeddings, retrieval and prompt engineering.
- LLM Frameworks – experience with LangChain, LlamaIndex or LangGraph, plus Azure OpenAI, OpenAI or Databricks Model Serving.
- Azure DevOps / CI-CD – experience automating data pipeline deployments; Terraform and Databricks Asset Bundles are highly relevant.
- Technical Leadership – ability to design solutions, write production code, review code and guide data engineering teams.
- Financial services or insurance data, especially financial reconciliation, general ledger or accounting data.
- AI-driven data quality, anomaly detection and automated reconciliation.
- Knowledge graphs, ontologies, semantic layers or enterprise Text-to-SQL solutions.
- Azure security, data governance, PII protection and fine-grained access controls.
- MLOps/LLMOps, model evaluation and production AI monitoring.
Important: This is a hands-on Lead Data Engineer position, not a purely managerial or coordination role. Candidates must demonstrate recent production coding experience.
4. Recruiter Screening Checklist
Confirm all of the following before submission:
- 12–18 years of relevant experience.
- Visa status and eligibility to meet the applicable compensation terms.
- H-1B transfer candidate, if applicable, can start upon receipt of the transfer receipt notice.
- Willing and able to work onsite in Cary, NC, from Day 1, Tuesday–Thursday.
- Strong hands-on Python, PySpark, Scala and SQL.
- Strong Azure Databricks, Delta Lake, ADLS Gen2 and ADF experience.
- Proven enterprise lakehouse / medallion architecture experience.
- Production experience building and deploying LLM/RAG solutions.
- Recent production coding and technical leadership experience.
- Compensation expectations confirmed before submission.
Candidate Priority
Highest priority: Candidates who combine Azure Databricks and enterprise data engineering expertise with production LLM/RAG implementation experience.
Secondary priority: Candidates with strong data platform engineering and production AI experience who also bring insurance/financial services domain knowledge and governance expertise.
Do not prioritize: Pure people managers, candidates without hands-on production AI/LLM experience, candidates lacking core Azure Databricks expertise, or candidates unable to meet the mandatory onsite and compensation requirements.
