We are building AI-native solutions for our clients - products where LLM and its harness are the core of the value. This is a builder's role: you and your team are responsible for building agentic systems, writing the production code, and standing up the evals and observability. You will work closely with SMEs and end-users to understand where the real value lies, and you design the feedback loops. Responsibilities Design, build and ship AI-native systems E2E - agents, workflows, RAG and the harness: custom tool calling, sandboxing, context engineering and sub-agents, caching, compaction Build the evaluation pipelines and use them to prove the system is genuinely useful Design for failure in the agent loop: retries, model fallbacks, cost limits and human-in-the-loop on consequential actions Capture domain expertise and repeatable workflows so what works on one engagement carries to the next Engage early to help shape the use case and check technical feasibility Write production-grade Python: integrations, APIs, data access, deployment Work directly with SMEs and end-users through interviews, UAT and observing the real workflow, and validate that the system fits how people actually work Requirements 7+ years of engineering experience, with a strong recent track record building production AI / LLM applications (not prototypes or research only) Strong agent-design judgment - task-harness fit, matching the harness to the context, failures and policies of the actual task rather than calling a model in a loop Capability to operate close to the client: lead discovery and feasibility conversations, work directly with SMEs and end-users, and explain technical trade-offs to both technical and non-technical audiences Hands-on experience with agentic frameworks (LangChain, LangGraph, Semantic Kernel) and major LLM providers (OpenAI, Anthropic, Google Gemini) Expert-level Python and solid software engineering fundamentals Strong RAG and retrieval skills: vector databases, embeddings, hybrid search, re-ranking, chunking and context management Proven experience evaluating generative AI quality - LLM-based evaluation, heuristics, custom eval frameworks - and using observability/tracing tools (LangSmith, Arize Phoenix, Langfuse) Production deployment experience on at least one major cloud (AWS, Azure, Google Cloud Platform) with containerization, CI/CD Sound judgment under ambiguity - scoping, sequencing and making the call on speed vs. quality vs. scope English at C1 level Nice to have Experience designing experiments, A/B testing and iterating on AI products against real user behavior and business metrics Background in NLP, Data Science or applied ML, with experience moving models into production Familiarity with MCP, A2A and Agent Skills, and emerging agent standards Experience with enterprise AI platforms (AWS Bedrock AgentCore, Databricks Genie, Microsoft Foundry) Exposure to AI governance, security and compliance (guardrails, prompt-injection prevention)
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
- Dice Id: 10330481
- Position Id: b968f86984ccff4c1c8d403142b616f9
- Posted 30+ days ago
