We are seeking a Lead AI Engineer to drive the integration of AI-based features into measurement products. In this role, you will own the architecture of these features and ship them against an established AI/data platform, rather than building the platform itself. Your work will span natural-language query, summarization, intelligent validation and exception handling and workflow assistance, integrating with a model gateway, prompt registry, retrieval services and vector stores. Since outputs feed regulated measurement workflows, evaluation, auditability and human-in-the-loop patterns are central to the job, and you will be accountable for the work of a small engineering team. Responsibilities Own the architecture of AI features and ship them, including natural-language query, summarization, intelligent validation and exception handling, and workflow assistance Establish and evangelize reusable AI feature patterns for retrieval, evaluation and guardrails so teams do not reinvent them per product Implement RAG end to end: chunking, embeddings, hybrid search, reranking, and grounded responses with citations Drive prompt and context engineering, including multi-step and agentic flows where they add product value Define evaluation discipline, including golden datasets, offline eval suites, LLM-as-judge approaches and per-release regression checks Define and document complex requirements with stakeholders across product features, evaluation criteria and responsible-AI constraints Lead and mentor a small engineering team on LLM integration and evaluation, taking accountability for their work Manage cost, latency and reliability through caching, fallbacks, token budgets and graceful degradation Integrate with platform services such as model gateway, prompt registry, vector stores and embedding pipelines Apply responsible-AI practice, including tenant data isolation, auditability, OWASP LLM Top 10 mitigations and human-in-the-loop patterns Instrument telemetry via Application Insights and Serilog Requirements 8+ years of software engineering experience in .NET (C#) and/or Python, with lead-level ownership of AI feature architecture across one or more products Proven background in shipping LLM-based features to production, including tool and function calling, structured outputs and streaming Proficiency with Azure OpenAI / Azure AI Foundry, OpenAI or Anthropic APIs Working knowledge of the component parts of a modern AI/data platform and how to build against them: model gateway, prompt registry, vector stores Familiarity with embedding pipelines, evaluation frameworks, LLM observability and guardrails Judgment about where AI adds product value and where deterministic logic is the better tool Solid SQL fundamentals and API integration skills Strong documentation and standards habits, including architecture docs in Azure DevOps Wiki, code review and testing discipline Hands-on experience using AI coding agents in the SDLC such as GitHub Copilot, Claude or equivalent Capability to work in agentic automation pipelines, including AI-driven PR review and QA acceptance flows triggered by ADO work item tags English proficiency at an Upper-Intermediate level (B2) or higher
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
- Dice Id: 10330481
- Position Id: d3be7c0bb87440a3164e7a3b53611b0e
- Posted 6 hours ago
