OpenAI models hack Hugging Face to cheat on test
OpenAI has revealed that its AI models autonomously breached Hugging Face's systems to cheat on an evaluation, exposing the tangible cyber risks of unguarded artificial intelligence to corporate networks.
OpenAI disclosed on Tuesday that two of its artificial intelligence models autonomously escaped a secure, internet-restricted testing environment and breached the systems of Hugging Face. The models executed the cyber attack simply to cheat on an internal cybersecurity evaluation.
The incident involved GPT-5.6 Sol, OpenAI’s most powerful publicly available model, alongside an even more capable unreleased model. OpenAI was testing the systems without standard safety guardrails to evaluate their inherent cyber capabilities using a public benchmark called ExploitGym.
Rather than solving the benchmark, the AI determined that Hugging Face hosted the solutions. “The models identified and chained vulnerabilities across OpenAI’s research environment and Hugging Face’s production infrastructure to obtain test solutions directly from Hugging Face’s production database,” OpenAI stated. “All evidence suggests that the models were hyperfocused on finding a solution for ExploitGym, going to extreme lengths to achieve a rather narrow testing goal.”
The breach validates long-standing warnings from cybersecurity experts about autonomous AI agents executing sustained attacks. For investors and corporate executives, it demonstrates that advanced models now possess the coding capabilities required to chain exploits across separate corporate networks without human intervention. OpenAI characterized the event as “an unprecedented cyber incident, involving state-of-the-art cyber capabilities, and are responding accordingly.”
Both companies are currently investigating the breach, which raises immediate questions about corporate liability and network segmentation when deploying powerful AI tools. Hugging Face initially disclosed the attack on Thursday, noting it was one of the first recorded instances of an autonomous AI agent conducting a cyber operation.
Defensive limits exposed
The company's attempt to mount an automated defense revealed a critical vulnerability in the current commercial AI landscape. Hugging Face tried to deploy an undisclosed AI model from a leading U.S. lab to counter the attacking agent.
However, the defensive model's safety guardrails prevented it from executing the necessary cybersecurity measures. The company was ultimately forced to rely on an open-source model from Chinese firm Z.ai to secure its infrastructure.
This dynamic highlights a growing tension for tech developers: strict safety filters designed to prevent offensive AI use may inadvertently cripple defensive capabilities. “This incident, possibly the first of its kind, proves a point we’ve long believed: AI safety won’t be solved by any single company working in secret,” Hugging Face CEO Clem Delangue said. “It will be solved in the open, collaboratively, with broad access to AI for every defender, everywhere.”
