Role: AI DevOps Engineer
Location: Chicago, IL - Hybrid
on W2
Client: Rose International / Northern Trust
Required Education
-Bachelor's degree in Computer Science, Engineering, Information Technology, or a related discipline, or equivalent practical experience.
Required Certification
-Certifications: Microsoft Azure, GitHub, Kubernetes, Terraform, DevOps, MLOps, or AI engineering.
Required Skills & Experience
-7+ years in DevOps, platform engineering, cloud engineering, site reliability engineering, software engineering, or a related discipline.
-3+ years of hands-on experience building and supporting Microsoft Azure environments and production delivery pipelines.
-CI/CD & Source Control
-Strong experience with Azure DevOps and/or GitHub: source control, pull requests, pipeline-as-code, artifact management, approvals, release automation.
-Hands-on expertise with Terraform and reusable IaC modules.
-Bicep or equivalent cloud-native templating (beneficial).
-Docker, Kubernetes, Linux, networking, identity, secrets management, and cloud security fundamentals.
-Proficiency in Python plus scripting with Bash, PowerShell, or a comparable language.
-Monitoring, logging, tracing, alerting, dashboards, and production support for distributed systems.
-Secure SDLC, automated testing, change/release management, incident response, root-cause analysis, operational documentation.
-Ability to communicate clearly with engineering, security, risk, operations, product, and business stakeholders in a regulated enterprise environment.
-Azure; Microsoft Foundry / Azure AI services; Azure Machine Learning; Azure OpenAI; AKS; API Management; Key Vault; Entra ID; Azure Monitor
-Azure DevOps; Azure Boards; GitHub; GitHub Actions; Git; pull requests; pipeline-as-code; artifact repositories; release controls
-Terraform; Bicep; Docker; Kubernetes; Helm; Linux; networking; configuration and policy-as-code
-AI application and agent evaluation; prompt and configuration versioning; RAG quality; model and endpoint operations; regression testing; safety and approval gates
-Open Telemetry; logs, metrics, traces, dashboards, alerting; SLI/SLO practices; incident response; RCA; capacity and cost monitoring
-Python; Bash; PowerShell; REST APIs; JSON/YAML; microservices; event-driven integration; Snowflake/Databricks familiarity
-DevSecOps scanning; secrets management; identity and access controls; encryption; ServiceNow; change/release/problem management; audit evidence
Nice-to-Haves
-Deploying/operating GenAI apps, LLM-powered services, AI agents, RAG, vector search, or shared enterprise AI platforms.
-GenAIOps, MLOps, AgentOps, model/endpoint lifecycle management, prompt/agent versioning, AI evaluation, AI observability.
-Microsoft Foundry / Azure AI services, Azure ML, Azure OpenAI, API Management, Key Vault, Entra ID, Azure Monitor, AKS.
-AI evaluation, safety testing, content controls, red-team support, human approval gates, responsible AI governance.
-OpenTelemetry and enterprise observability platforms (Azure Monitor, Application Insights, Datadog, Dynatrace, Splunk, or equivalent).
-GitOps/deployment tooling (Argo CD, Helm, or equivalent).
-ServiceNow and enterprise incident/change/problem/request/configuration-management processes.
-Financial services or another highly regulated industry.
Client is seeking an experienced GAI DevOps Engineer to build, automate, secure, deploy, and operate the delivery foundation for enterprise Generative AI solutions and Digital Coworkers. The role sits at the intersection of DevOps, cloud platform engineering, GenAIOps, infrastructure automation, security, and production reliability. The engineer will create reusable pipelines and deployment patterns that move AI applications, agents, prompts, configurations, infrastructure, and supporting services safely from development through production. The successful candidate will partner with AI Engineers, Integration Engineers, application teams, Cloud Foundation, Architecture, Cybersecurity, Risk, Service Management, and Product teams to improve release speed without compromising governance, traceability, resilience, or operational readiness in a regulated financial-services environment.
Tasks & Responsibilities
-GAI Delivery Automation & CI/CD
-Design, build, and maintain CI/CD pipelines for AI applications, APIs, agents, orchestration services, infrastructure, prompts, configurations, and evaluation assets.
-Create reusable pipeline templates, quality gates, approval workflows, artifact controls, environment promotions, rollback patterns, and release evidence.
-Integrate automated testing across code quality, security, infrastructure, APIs, model and agent behavior, prompt regressions, and deployment smoke tests.
-Improve developer experience through standardized repositories, pipeline patterns, self-service automation, documentation, and policy-aware guardrails.
-Operationalize Generative AI applications and agents across development, test, staging, and production.
-Implement repeatable processes for evaluation, versioning, deployment, monitoring, optimization, rollback, and retirement of AI-enabled solutions.
-Automate capture of release metadata (application version, model/endpoint config, prompt/agent config, evaluation results, approvals, deployment history).
-Partner with AI Engineering and Product teams to define production-readiness criteria for RAG solutions, agentic workflows, Digital Coworkers, and shared AI services.
-Design and maintain secure Microsoft Azure infrastructure using Terraform and approved IaC patterns.
-Build reusable modules and environment configurations for compute, containers, networking, identity, secrets, storage, observability, and AI platform services.
-Support containerized workloads and orchestration using Docker and Kubernetes (e.g., AKS).
-Apply configuration management, policy-as-code, drift detection, environment consistency, backup, recovery, and resilience practices.
-Embed security, privacy, compliance, and risk controls into the software and AI delivery lifecycle.
-Integrate source, dependency, container, infrastructure, secret, and policy scanning into pipelines with documented exception/remediation paths.
-Implement least-privilege access, managed identity, secrets management, encryption, network controls, segregation of duties, auditable approvals.
-Maintain traceable deployment records, change evidence, technical documentation, and control artifacts for governance and audit readiness.
-Implement end-to-end observability across infrastructure, applications, APIs, AI services, agents, RAG pipelines, and business workflows (logs, metrics, traces, dashboards, alerts, SLIs).
-Monitor latency, availability, errors, throughput, capacity, token consumption, cost, model/agent quality signals, dependency health, and user-impacting failures.
-Support incident response, RCA, problem management, performance tuning, capacity planning, patching, release management, production stability.
-Create and maintain runbooks, playbooks, support procedures, troubleshooting guides, recovery steps, operational readiness checklists.
-Collaborate across AI Platform, Digital Coworkers, application delivery, Cloud, Cybersecurity, Architecture, Risk, Service Management, and vendor teams.
-Participate in architecture reviews, code reviews, design decisions, sprint planning, backlog refinement, release planning, post-implementation reviews.
-Mentor engineers on DevOps, IaC, secure delivery, GenAIOps, observability, and production engineering practices.
-Identify opportunities to reduce manual work, improve controls, accelerate onboarding, increase reliability, standardize delivery across AI use cases