Engineers wary of recommendations built on incomplete context and unreliable telemetry want better operational data, continuous monitoring and evidence they can independently verify.
Enterprise IT organizations are no longer debating whether to use AI, but rather which systems AI should influence, how much authority it should receive and whether engineers can trust its recommendations when production services are at stake.
Data collected by Netscout from 950 IT professionals found 82% of organizations are discussing, testing, planning or running AI- and large language model-based IT projects. However, that activity has not translated into equivalent confidence among the engineers responsible for operational outcomes.
When asked about obstacles to AI-driven IT operations, 22% said AI outputs could not be trusted, while 22.1% cited poor data quality or incomplete context. More than a quarter selected every listed operational challenge, including data silos, telemetry noise and mistrust.
“AI can do more of the work, but it does not take responsibility for the outcome,” explains Shams Chauthani, chief technology officer at Tempo Software.
Context Is the First Failure Point
An AI system can generate a technically plausible answer while missing the operational history, dependencies and constraints that determine whether the answer is useful. This is especially dangerous when the system presents an incomplete conclusion with high confidence.
“Poor data can lead it toward the wrong conclusion, while missing context or fragmented telemetry can prevent it from seeing relationships and constraints that would impact its recommendation,” says Dustin Johnson, CTO and founding partner at Seeq.
He adds the tricky part is that an incomplete picture doesn’t always prevent the AI from producing a confident answer.
The challenge extends beyond cleaning up datasets. Enterprise environments contain years of architectural decisions, undocumented dependencies and exceptions developed in response to earlier incidents.
An AI system examining an isolated component may not understand why it was configured in a particular way or what else will break if it changes. Maintenance work therefore places different demands on AI than building a new application.
Chauthani says an agent may understand the code immediately in front of it while missing decisions made years earlier or assumptions embedded elsewhere in the product.
“That is why a change can look perfectly reasonable in isolation and still be wrong for the system,” he says.
Telemetry quality presents another obstacle. Sampled or fragmented data can hide dependencies, generate false positives and lead AI toward the wrong root cause.
“If AI uses data trained on fragmented, sampled, noisy or incomplete telemetry, it can produce plausible answers without sufficient operational context,” says Phil Gray, AVP leading observability and AIOps product management at Netscout.
For IT professionals, this makes data literacy and systems knowledge increasingly important: Engineers must understand not only how models work but also whether the inputs accurately represent production conditions.
Evidence Must Precede Autonomy
Trust should depend on what an AI system can demonstrate, not how confidently it communicates. Engineers need to know what the system observed, which assumptions it made and why it recommends a particular action.
“Engineers should be able to see all the evidence behind an AI system’s recommendation--what data the AI used, what assumptions it made and how confident it is in the conclusion,” Johnson says.
The evidentiary standard should rise with the potential impact. For example, an AI system flagging unusual latency for investigation does not require the same controls as one that can modify network configurations, restart services or deploy code without approval.
Gray says recommendations should be traceable to observed behavior and explain what happened, where it occurred, which users or services were affected and why the proposed response is appropriate.
“The greater the autonomy, the stronger that evidentiary foundation must be,” he says.
For AI-generated software changes, engineers also need records showing what the agent was asked to accomplish, what context it accessed, which files and services it changed and what tests it performed.
Observe Decisions and Outcomes
The survey also found nearly two-thirds of respondents consider the ability to monitor AI processes and services valuable or critical. That monitoring must extend beyond model availability and response time to include the actions AI takes and their effects on surrounding infrastructure.
Johnson says engineers need enough historical visibility to reconstruct what an AI system saw, recommended and did, followed by what happened as a result. That requires logging inputs, contextual information, recommendations, actions and boundary violations.
“This kind of auditability gives teams something concrete to evaluate when determining if a system should be given more trust and autonomy,” he says.
Independent operational evidence is particularly important when teams need to verify an AI-generated explanation. Network data can reveal how applications and services communicated, helping engineers confirm whether the model identified the correct dependency or root cause.
However, less than one in 10 of the IT pros surveyed said deep packet inspection was their primary observability data source, even though 64% considered it important or very important.
It’s a gap that suggests many organizations recognize the value of granular network evidence but have not integrated it into their monitoring strategies.
“Organizations need continuous visibility into both AI recommendations, actions and the infrastructure those actions affect,” Gray says. “This creates accountability and allows teams to determine whether automation improved, or inadvertently disrupted, a service.”
Skepticism Becomes a Career Asset
As AI assumes a larger role in IT operations and software development, the ability to evaluate its work will become an increasingly valuable professional skill. Engineers do not need to abandon skepticism; they need methods for converting it into measurable evidence.
That will require stronger data and telemetry literacy, along with the ability to identify missing operational context.
Engineers must know how to design repeatable evaluations for AI-generated work, trace recommendations through logs and network activity, assess blast radius and reversibility, and create audit trails that show when human approval was obtained.
They must also be able to turn incidents, rollbacks and other failures into tests that improve the system over time.
Organizations can begin with bounded, low-risk tasks whose outcomes are easy to measure. If an AI system performs consistently, teams can gradually reduce human intervention for that specific category of work while retaining stronger controls elsewhere.
This approach gives engineers a central role in AI adoption, where their skepticism helps define the monitoring, tests and operational boundaries that make greater automation possible.
“You do not decide that you trust an agent everywhere,” Chauthani says. “You build evidence that it can handle specific kinds of work, and you expand from there.”
