AI Agent Developer Microsoft Copilot Studio

Overview

Remote
Depends on Experience
Contract - W2
Contract - Independent

Skills

Copilot Studio
Azure
CI/CD
GitHub

Job Details

  • Experience Level:8+ years overall, 3+ years in Generative AI / LLM solutions, 1+ year hands-on with Microsoft Copilot Studio building and operationalizing agents in production

Position Overview

We are looking for a hands-on architect-developer who can stand up Microsoft Copilot Studio platform, design repeatable patterns for AI Agent development, and lead the full lifecycle from ideation through production deployment. You will be the first dedicated Copilot Studio specialist on the team, partnering with other teams to build governed, observable, and cost-optimized agents that deliver immediate business value and scaling to an enterprise platform.

Key Responsibilities

Area

What You ll Do

Architecture & Platform Setup

Stand up Copilot Studio and Microsoft AI Foundry environments (networking, security, RBAC, data connections, CI/CD).
Establish reference architectures, naming standards, and configuration baselines for new copilots.

Agent Development

Design, prompt-engineer, and build AI Agents in Copilot Studio to ingest, reason over, and act on large document corpora (SharePoint, email, insurance policies, safety plans).
Implement seamless push / pull pipelines between Copilot Studio and AI Foundry.

Dev Test Prod Automation

Create repeatable promotion pipelines with version control, automated testing, and rollback strategies.
Define branching and release strategies that fit SDLC.

Observability & Monitoring

Instrument copilots with robust logging, telemetry, and usage analytics (quality, latency, cost per request).
Build dashboards / alerts to track agent health, hallucination rates, accuracy, response times, and user satisfaction.

Governance & Cost Optimization

Enforce data-access controls, privacy policies, and prompt-safety gates.
Tune prompts, embeddings, and capacity planning to minimize Azure consumption without compromising quality.

Knowledge Transfer & Leadership

Mentor developers and automation engineers; run working sessions, share best practices, and codify lessons-learned into internal playbooks.

Must-Have Qualifications

  • Proven production experience building Copilot Studio agents end-to-end (prompt design deployment monitoring).
  • Strong grasp of AI Foundry, vector stores, Retrieval-Augmented Generation, and grounding strategies.
  • Expertise in prompt engineering, evaluation techniques (BLEU, semantic similarity, human-in-the-loop).
  • Fluency with Azure DevOps / GitHub Actions for CI/CD, Infrastructure-as-Code (Bicep/Terraform/ARM, or similar).
  • Implemented observability stacks (App Insights, Azure Monitor, OpenTelemetry, Log Analytics, KQL, or similar).
  • Demonstrated cost-management of GenAI workloads (token budgeting, capacity reservations, caching).
  • Excellent communication skills; able to translate technical concepts for business stakeholders and teach peers.

Nice-to-Have

  • Experience integrating Microsoft Purview for policy enforcement and data-lineage of LLM pipelines.
  • Familiarity with SharePoint, Exchange / Graph API, and insurance domain terminology (policy, loss control).
  • Hands-on with LangChain, Semantic Kernel, or similar frameworks but comfortable building natively in Azure.
  • Experience with multi-models including GPT4, GPT 4o, GPT 3.5, o3, Llama, and other models.

Pardha S S Moyida

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.