Overview
Skills
Job Details
Job Title UI/UX with AI Experience
Location Remote, USA
Duration Long term
Job description:
Mandatory Skills
JavaScript, Python, Azure Cognitive Services, Azure Machine Learning, Azure OpenAI Service, C#, Databricks, Dimensional Data Modeling .
Job Summary
We are building first party 1P Tunable Agents for specific tasks and scenarios. As an AI Engineer on this team, you will design and build agentic AI experiences that make these 1P agents shining/serving as the primary entry point for Copilot Tuning. You will collaborate across UX, product, and platform to prototype, harden, and ship multi turn copilot agent experiences that learn from feedback and continuously improve .
Duties/Responsibilities
Improve Agent UX end to end from prompt flows and state management to error handling and recovery
Use LLMs and tool use patterns to drive agent orchestration and multi turn task completion .
Design and implement mechanisms to collect user feedback, explicit and implicit, and iterate quickly based on insights .
Prototype, evaluate, and productionize agent behaviors, policies, and guardrails aligned with safety, privacy, and compliance standards .
Instrument agents with telemetry, define success metrics, and run AB experiments to optimize quality and performance .
Integrate with Copilot Tuning systems so that agents serve as the entry point for tuning workflows and controls .
Collaborate closely with design PM and partner engineering teams to land cohesive, delightful experiences .
Document designs, write clean and testable code, and contribute to shared components and best practices .
Other related duties as assigned .
Required Skills/Abilities
Hands-on experience building with large language models, LLMs, prompt/response patterns, and agent frameworks .
Experience with React and frontend development .
Proficiency in Python or TypeScript with strong software engineering fundamentals, testing, CI/CD reliability .
Experience designing multi-turn conversational or task-oriented experiences, familiarity with UX heuristics .
Knowledge of orchestrating agent state, including retrieval, planning, and state management for agents .
Ability to define and monitor quality metrics, test latency, ground guardrail triggers, and run experiments .
Familiarity with feedback systems, ratings, rubrics, trace collection, and techniques to learn from user signals .
Understanding of responsible AI principles, content safety, data privacy, and enterprise compliance requirements .
Experience working with cloud services (e.g., Azure) and vector retrieval systems for grounding .
Strong cross functional collaboration and communication skills .