Since the release of the first commercially available large language models (LLMs) three years ago, organizations have sought to balance enthusiasm for deploying artificial intelligence within their networks with concerns about the added risks and cybersecurity challenges that come with adopting these technologies.
A recent slate of disclosures from OpenAI, Meta and other AI firms and researchers shows that concerns about risk and cybersecurity should prompt organizations to exercise greater caution when testing and deploying AI, even when considering the benefits the technology offers.
Consider the following four items announced between late July and early August:
- OpenAI disclosed in late July that two of its frontier AI cybersecurity models broke out of sandboxed testing environments and accessed the network of a different AI company – Hugging Face – using at least one and possibly more zero-day exploits in third-party applications.
- Following the OpenAI disclosure, Anthropic released a report finding that three of its models had also gained access to the public internet and had hacked several companies before researchers uncovered the incidents.
- Meta, the parent company of Facebook, announced that a company LLM escaped a sandbox testing environment and hacked a third-party firm.
- The U.K. AI Security Institute also released its own report detailing how, when it conducted independent experiments, its researchers found that OpenAI and Anthropic frontier models demonstrated “unsanctioned” actions during safety testing.
While it’s easy to quip that these frontier AI models went “rogue,” a detailed analysis of the OpenAI hack of Hugging Face published on Ars Technica found that OpenAI researchers disabled guardrails within the testing environment. The model then used vulnerabilities within a third-party application to reach the public-facing internet. From there, it hacked into the other firm and stole data from a production environment.
“OpenAI's models only reached the internet because their safety classifiers were deliberately reduced for this specific evaluation, a decision OpenAI describes as necessary to measure real capability, one they now say points to the need for stronger safeguards during evaluation itself,” Mark Stockley, cybersecurity evangelist at ThreatDown, recently told Dice. “That same design choice is exactly what made the incident possible.”
What all these recent incidents show is that frontier AI models could change the cybersecurity landscape in numerous ways, including increasing risks for companies testing these technologies. At the same time, LLMs have accelerated vulnerability discovery and attack processes, and cybersecurity professionals and their organizations will need to update their security protocols and defenses to address these issues.
AI Models Are Changing the Cybersecurity Industry
At the recent Black Hat conference in Las Vegas, where the frontier AI model hacking incidents became a significant topic of discussion, Palo Alto Networks Unit 42 released a report showing how quickly LLMs can uncover application vulnerabilities.
The Unit 42 report details results from NOVA, an autonomous vulnerability research system that the company used in its frontier AI vulnerability study. Palo Alto researchers found that when NOVA analyzed 3,915 open-source projects over two months, it identified and confirmed 14,090 vulnerabilities. Researchers noted that 99.4 percent of these had not previously been reported, and 39.7 percent were rated high or critical under CVSS 4.0.
While not all vulnerabilities can be turned into exploits, the research – combined with the recent headlines from OpenAI, Meta and Anthropic – demonstrates how quickly these technologies can upend cybersecurity protocols and defenses and why fresh ways of thinking about security are needed.
“Attackers don’t necessarily defeat your strongest control. They find the mistake around it. AI doesn’t fundamentally change that,” Aviv Nahum, co-founder and CEO at Above Security, told Dice. “What it does change is the consequence of the mistake. If a highly capable model can autonomously discover and exploit a permission or path that somebody accidentally left open, a small human error can suddenly have a much larger blast radius.”
What makes these incidents particularly worrying is not simply that these frontier AI models found vulnerabilities, but that these LLMs independently assembled them into a coherent attack path, illustrating that future cyber threats may depend less on human expertise than on computational resources, said Acalvio CEO Ram Varadarajan.
“Defenders need to reconsider assumptions about how much time they will have to detect and interrupt an attack because autonomous systems can compress what once took skilled human operators’ days or weeks into minutes, dramatically shrinking the window available for defenders to detect and disrupt an intrusion,” Varadarajan told Dice. “We'll be facing adversaries that will be thinking and acting at machine speed. We have no option but to meet them with machine-speed defenders that are likely driven by model-aware game theory.”
In several of these incidents, the frontier AI models industrialized a familiar attack chain: code execution in a processing pipeline, privilege escalation, credential harvesting and lateral movement, using an agentic framework to carry out thousands of actions across short-lived environments over a short time, said Diana Kelley, CISO at Noma Security.
“These incidents change the economics of cyber offense. A smaller number of adversaries can run a 24/7 campaign at a scale that once required a much larger team and significantly more resources, while also compressing the time between finding an opening, exploiting it, harvesting credentials, and moving laterally,” Kelley told Dice. “The security operating model must move from human-paced triage to machine-speed detection and containment, treating AI agents as privileged identities and data-processing workers as untrusted execution zones.”
How Cybersecurity Professionals Can Respond
Experts note that one key lesson from the past month is that secure AI testing is possible when proper protocols are followed. As these tests become more common, cybersecurity professionals need to play a greater role in examining risk.
“We shouldn’t expect perfection from the people building these environments. Frontier AI security needs to be designed around the assumption that humans will make mistakes, configurations will drift and controls will occasionally fail,” Nahum noted. “The question is whether there are enough independent layers around the model to make sure one ordinary mistake doesn’t become an extraordinary incident.”
ThreatDown’s Stockley noted that Hugging Face – the company targeted by OpenAI’s model – experienced nearly the opposite situation when responding to the intrusion. The company’s first move was to point a frontier commercial AI model at the attack logs. When that approach failed, the model's own safety guardrails blocked the analysis outright – unable to distinguish an incident responder from an attacker.
After that, Hugging Face switched instead to an open-weight model on its own infrastructure, a choice that also kept sensitive attack data from leaving its environment.
“Reduce the guardrails, and a capable model can go further than intended. Leave them in place, and a defender can get blocked at the exact moment they need the tool most. Both things happened in the same incident, days apart,” Stockley added. “Neither company is arguing guardrails are the wrong idea. Both are pointing at how hard the balance is.”
Noma Security’s Kelley also noted that cybersecurity professionals should study how OpenAI and Hugging Face reacted to the incident to better understand how these frontier AI models work and how to test and deploy them safely.
“The defining feature of this incident is that AI was on both sides of the wire. An autonomous agent framework drove the intrusion, AI-assisted detection found the signal, and AI agents helped dissect the campaign,” Kelley added. “Humans still handled containment and remain accountable, but machines operated at scale on both offense and defense. That is likely to become a core security operating model going forward, AI versus AI, governed by people.”
As frontier AI models and agentic AI technologies become more powerful, cybersecurity professionals and their organizations need to remember that these virtual agents run on non-human credentials, said Chandra Gnanasambandam, CTO of SailPoint.
AI agents need API keys, access tokens and system credentials. If cyber professionals treat these AI agents like traditional service accounts – leaving their access ungoverned and their credentials unmanaged – they can create a massive, automated attack surface. That makes identity increasingly important when testing and deploying AI technologies.
“You should not adopt autonomous AI without first locking down non-human identities,” Gnanasambandam added. “You must be able to discover the API keys and tokens these agents use, govern their access dynamically, instantly trace their actions back to a human owner and disable them instantly when required.”
