Role: Agentic AI & Observability Engineer
Location: Plano, TX or Mc Lean VA Prefer Local
Role Summary
We are looking for an experienced Agentic AI & Observability Lead or an Architect with strong hands-on expertise in AI agent development, MCP (Model Context Protocol) server development, observability, data analytics, and cloud-native engineering. The candidate will design and build intelligent agents, MCP-based integrations, telemetry and analytics solutions, and automated provisioning capabilities across modern distributed application environments.
The ideal candidate should have strong development experience in Python and/or JavaScript, combined with practical knowledge of OpenTelemetry, Kibana, distributed tracing, log analytics, Kubernetes, AWS, data streaming, CI/CD, and security & compliance.
Key Responsibilities
- Design, develop, and deploy AI/Agentic AI applications and autonomous agents that interact with enterprise systems, APIs, data platforms, and operational tools.
- Design and build MCP Servers to securely expose enterprise tools, APIs, data sources, and operational capabilities to AI agents.
- Develop reusable MCP tools, resources, and integrations, including authentication, authorization, error handling, logging, and monitoring.
- Build backend services, APIs, integrations, and automation using Python and/or JavaScript.
- Implement application and platform observability using the OpenTelemetry framework, including logs, metrics, and traces.
- Implement and troubleshoot distributed tracing across microservices and distributed application environments.
- Build advanced log analytics, dashboards, visualizations, and operational insights using Kibana.
- Analyze telemetry and operational data to identify anomalies, application failures, performance bottlenecks, and recurring operational patterns.
- Design and integrate real-time and near-real-time data streams for telemetry, analytics, and AI-agent use cases.
- Develop and deploy solutions in Kubernetes-based containerized environments.
- Design and integrate solutions with AWS services and cloud-native architectures.
- Build and maintain CI/CD pipelines for automated build, testing, security validation, and deployment.
- Automate application, infrastructure, and service provisioning using Python or JavaScript.
- Apply enterprise security, compliance, identity, access-control, secrets-management, and data-protection requirements across agent and observability solutions.
- Work with application development, SRE, DevOps, platform, security, and architecture teams to onboard applications and implement observability and AI-driven operational capabilities.
- Develop production-quality solutions with appropriate scalability, resiliency, performance, security, and monitoring.
Required Technical Skills
Agentic AI & Development
- Strong experience developing AI agents / Agentic AI solutions.
- Hands-on experience designing and building MCP Servers and integrating MCP clients, tools, resources, APIs, and enterprise data sources.
- Strong programming experience in Python and/or JavaScript.
- Experience developing REST APIs, backend services, integrations, and automation frameworks.
Observability & Analytics
- Strong hands-on experience with OpenTelemetry (OTel).
- Experience implementing and analyzing distributed traces, logs, and metrics.
- Strong experience with Kibana dashboards and log analytics.
- Ability to correlate telemetry across distributed systems for troubleshooting and root-cause analysis.
- Experience working with high-volume telemetry and data streams.
Cloud & Platform Engineering
- Strong experience with Kubernetes, containers, and microservices architectures.
- Hands-on experience with AWS and cloud-native services.
- Experience developing and maintaining CI/CD pipelines.
- Experience with infrastructure/application provisioning and automation using Python or JavaScript.
Security & Compliance
- Understanding of secure application and API development.
- Experience implementing authentication, authorization, RBAC, secrets management, encryption, and secure communication.
- Ability to develop solutions aligned with enterprise security, governance, audit, and compliance requirements.
Preferred Qualifications
- Experience integrating AI agents with observability, incident management, DevOps, or enterprise operational platforms.
- Experience building agent-based solutions for incident triage, root-cause analysis, log analysis, remediation, and operational automation.
- Knowledge of SRE, AIOps, SLIs/SLOs, reliability engineering, and automated remediation.
- Experience working with large-scale distributed and microservices-based enterprise applications.
- Understanding of secure AI-agent architectures, including tool permissions, guardrails, identity propagation, auditability, and controlled access to enterprise systems.
Never miss an opportunity! Create an alert based on the job you applied for.