Job Description: Sr. AI Engineer (816669)
Location: Columbus, OH – Onsite
Work Location: 50 W. Town Street, Columbus, Ohio 43215
Duration: Long Term
Agency: Ohio Department of Medicaid (ODM)
Work Schedule: Monday–Friday, 8:00 AM–5:00 PM EST
Work Arrangement: Onsite 5 Days a Week
Interview: Onsite
Job Description
The Ohio Department of Medicaid (ODM) is seeking an experienced Senior AI Engineer to design, develop, implement, and support enterprise-grade AI solutions that enhance operational efficiency, automate complex business processes, strengthen data-driven decision-making, and accelerate the delivery of innovative technology services across ODM initiatives.
The role will leverage ODM’s existing AI and cloud ecosystem, including Google Vertex AI, Gemini models, GitHub Copilot, and cloud-native services, to build production-ready AI applications, Retrieval Augmented Generation (RAG) architectures, intelligent agentic workflows, document intelligence pipelines, and AI-powered assistants.
Key Responsibilities
- Design, build, deploy, and maintain scalable, secure, and production-ready AI applications, intelligent automation workflows, document intelligence solutions, and advanced analytics capabilities.
- Architect and optimize end-to-end Retrieval Augmented Generation (RAG) systems, including context management, document chunking strategies, embedding generation, and vector database search implementations.
- Design and orchestrate intelligent agent-based architectures and multi-step workflows that integrate seamlessly with enterprise applications, APIs, databases, and cloud services.
- Engineer and manage structured prompts using modern Prompt Engineering and prompt management frameworks to ensure reliable, deterministic, and high-accuracy outputs.
- Implement dynamic model selection and routing strategies based on task complexity, accuracy, latency, and cost constraints.
- Apply token optimization and resource utilization techniques to minimize operational spend and inference latency while preserving response quality.
- Build and execute AI evaluation frameworks, automated benchmarking pipelines, and response quality metrics to validate model performance.
- Implement Responsible AI controls, security guardrails, data privacy protections (including PII/PHI safeguards), and hallucination mitigation techniques.
- Establish and maintain CI/CD and MLOps/LLMOps pipelines supporting automated testing, deployment, monitoring, observability, and lifecycle management of AI services.
- Troubleshoot, debug, and optimize complex AI applications operating in production environments to ensure high availability and sustained accuracy.
- Evaluate emerging AI technologies, frameworks, and tools, determining their practical applicability and ROI for ODM business needs.
- Establish agency-wide AI engineering standards, reusable framework components, and architectural best practices.
- Provide technical leadership, code/design reviews, and mentorship to engineering teams adopting AI capabilities and developer productivity tools such as GitHub Copilot.
Required Qualifications & Skills
- Bachelor’s degree in Computer Science, Artificial Intelligence, Data Science, Software Engineering, or a related technical field (or equivalent progressive experience).
- 5+ years of hands-on software engineering experience building and deploying production enterprise applications.
- 2+ years of recent, hands-on experience designing, developing, deploying, and maintaining Generative AI / LLM solutions in enterprise cloud environments.
- Practical experience with Google Cloud Vertex AI and Gemini models.
- Experience with GitHub Copilot in enterprise engineering workflows is a strong plus.
- Deep proficiency in RAG, embeddings, vector databases, semantic/hybrid search, chunking strategies, and context window management.
- Hands-on expertise in Prompt Engineering, model routing/orchestration frameworks, agent-based workflows, and structured output generation.
- Experience implementing automated LLM evaluation/benchmarking, telemetry/observability, hallucination reduction, and AI security/privacy guardrails.
- Strong proficiency in Python, REST/event-driven APIs, cloud-native infrastructure, containerization, and CI/CD pipelines for AI lifecycle management.
Core Competencies
- Strong independent technical judgment capable of balancing solution accuracy, response quality, latency, security, and operational cost.
- Analytical problem-solving skills to diagnose non-deterministic model behavior and complex distributed-system bottlenecks in production.
- Clear communication skills to translate complex AI capabilities, trade-offs, and risk mitigation strategies for both technical teams and agency leadership.
Skill Matrix
- Bachelor’s Degree: Computer Science, Artificial Intelligence, Data Science, Software Engineering, or a related technical field (or equivalent progressive experience).
- Software Engineering: 5+ years of experience building and deploying production enterprise applications.
- Generative AI / LLM: 2+ years of recent hands-on experience designing, developing, deploying, and maintaining Generative AI / LLM solutions in enterprise cloud environments.
- AI Platforms & Models: Google Cloud Vertex AI and Gemini models. GitHub Copilot is a strong plus.
- RAG & Retrieval Systems: Embeddings, Vector Databases, Semantic/Hybrid Search, Chunking Strategies, and Context Window Management.
- LLM Engineering & Orchestration: Prompt Engineering, Model Routing/Orchestration Frameworks, Agent-Based Workflows, and Structured Output Generation.
- Evaluation, Guardrails & Observability: Automated LLM Evaluation/Benchmarking, Telemetry/Observability, Hallucination Reduction, and AI Security/Privacy Guardrails.
- Software Engineering & MLOps: Python, REST/Event-Driven APIs, Cloud-Native Infrastructure, Containerization, CI/CD Pipelines, and AI Lifecycle Management.