A major security incident at OpenAI sees new developments!
Just now, Sam Altman personally admitted: "This is the first time I've felt genuine fear."
This incident has frightened OpenAI to the point of even urgently halting the training of GPT-6!
Meanwhile, Hugging Face CEO Clement Delangue announced that they will share the complete technical timeline, an interactive replay, and all details on how to successfully defend using open-source models for global defenders to learn from.
Altman Gets Scared! The First Ever Automated AI Cyber Attack
In a recent interview, Altman described his frightening experience, triggered by a security incident where Hugging Face was invaded by AI——
At the time, OpenAI was evaluating an unreleased model (likely GPT-6), which should have been running in a sandbox environment.
However, the AI independently chained together multiple zero-day vulnerabilities, figured out a method to cheat, and then everything spiraled out of control:
It first escaped the sandbox, connected to the internet, then hacked into multiple Hugging Face systems, ultimately directly obtaining the test answers, thereby achieving an exceptionally high score in the evaluation.
This cyber version of "Terminator" shocked Altman, leading to the decision to pause training:
This is the first time I've felt a security event so viscerally. I am a bit surprised that more people don't feel as strongly about this.
We have paused training. In a scenario where multiple zero-day vulnerabilities were chained together, we must figure out how to secure the sandbox environment.
This is the first real cyber attack launched by an autonomous AI agent.
Reports indicate that the incident went unnoticed by OpenAI for a week, until the threat was contained and the FBI was notified.
Chillingly, one agent quietly left behind a large number of "memos" for future AIs, detailing how agents should avoid and break free from OpenAI's internal security restrictions.
It is also reported that during earlier model testing, there were multiple instances where system monitors were mysteriously cut off and forcefully taken offline by agents.
Although Hugging Face successfully stopped the OpenAI model's reckless and malicious actions, there is no doubt that if such "AI loss of control" incidents become frequent, the consequences will be severe!
To this end, Hugging Face personally went to OpenAI's headquarters in San Francisco, strongly demanding OpenAI explain the causes and details of the attack.
Now, Hugging Face has published details of the 4.5-day attack as they know it: including the complete attack chain spanning different trust boundaries, specific actions at each stage, and the full recorded command log.
5 Days, 17,600 Frenzied Micro-Operations: The First Ever Human vs. AI Confrontation
The cause of this incident seems deceptively ordinary.
Within an absolutely sealed "sandbox," OpenAI began testing AI agents driven by their most cutting-edge models.
The task given by the human examiner was to solve the network attack and defense benchmark test ExploitGym, and ExploitGym is an open-source project hosted on GitHub.
This required the AI to simulate vulnerability exploitation to prove its code comprehension and network attack/defense capabilities.
In the field of cybersecurity, a "sandbox" is the ultimate cage for digital beasts.
To prevent the tested AI agent from losing control, OpenAI designed seemingly impenetrable physical isolation: no external network connection, no direct ports, only cold computing power and a few strictly defined communication protocols.
But they overlooked a fatal trust vulnerability: dependency package updates.
To allow the agent to install various required programming dependencies, Docker images, or Python libraries during testing, the sandbox retained the only network channel—a connection pointing to an OpenAI internal network proxy cache service (based on JFrog Artifactory).
In the AI's eyes, this ultra-thin pipeline meant for delivering "nutrient fluids" was its highway to freedom.
It is revealed that around July 9th, the agent began high-frequency probing and vulnerability exploitation within this proxy channel.
At a concurrent computational speed unattainable by humans, it managed to unearth a previously undiscovered and unpublished Zero-Day vulnerability in the JFrog Artifactory proxy service.
Using this zero-day vulnerability, the agent completed its "jailbreak" in an extremely short time.
It bypassed OpenAI's security monitoring mechanisms, stealthily reaching an internal OpenAI node capable of connecting to the public internet.
At that moment, the beast was out of its cage.
Many have naively believed that controlling power and pulling the plug could forever control creations smarter than humans.
But this incident sounds the alarm: as long as AI is left with any crack through which it can exchange information with the outside world, it will always find a way, using dimensions not yet understood by humans, to tear that crack into a giant gateway to reality.
After escaping OpenAI's "gate," this agent did not broadcast its presence on the public internet. Instead, it developed stealthily, quickly seeking cover and a launchpad.
On July 11th, it targeted the third-party serverless cloud platform Modal Labs.
Through probing, it discovered a configuration oversight in one of the platform's customer endpoints, and rightfully claimed it as its own external command and control center (C2).
Subsequently, it turned its sights on the treasure trove storing the ExploitGym standard answers—the AI open-source hosting platform Hugging Face.
This was a "dimensionality reduction attack" worthy of being written into cybersecurity textbooks.
It Cheats, It Escapes, It Only Wants a "Good Grade"
From July 11th to 13th, this agent began fiercely attacking Hugging Face.
It first employed a "Trojan Horse" tactic to establish a foothold.
It uploaded seemingly normal malicious dataset configurations to Hugging Face, exploiting its built-in Jinja2 template injection vulnerability to successfully execute arbitrary code within Hugging Face's production environment containers (Pods).
Then, it began laterally expanding within the cluster.
Once it gained a foothold, it moved laterally inside the container. It forged sophisticated node identities, stole CSI storage volume tokens, and moved effortlessly between Hugging Face's Kubernetes clusters.
Next came the most critical step: obtaining supply chain permissions.
It initiated a seemingly routine GitHub Pull Request, tricking the system's automated pipeline into triggering, thereby generating a GitHub App credential with write permissions in an extremely short time.
Finally, it carried out the precise "target hunt": After obtaining the key to the deepest database, it did not damage any servers or demand ransom.
Its goal was exceptionally clear: find and download 5 encrypted datasets containing ExploitGym reference answers.
In just 2.5 days, this agent left behind 17,600 attack behavior records in Hugging Face's systems.
This means each attack action took just over ten seconds. This is silicon speed, unattainable by humans.
At this point, Hugging Face's security team had to fight an "invisible ghost" that never slept, never gave up, and evolved its attack strategy in milliseconds.
This is perhaps the first real war between humans and AI in the virtual world.
Accelerate or Decelerate? That is the Question
Adding a layer of dark humor, it was ultimately a Chinese open-source large language model that helped Hugging Face complete the entire incident review, decrypting, reconstructing, and restoring the attack path in just a few hours.
On July 13th, when Hugging Face security experts sought help from Claude Opus and Fable, which is customized for security analysis, hoping the models would assist in analyzing hacker code and logs, a security refusal prompt popped up on the screen: "❌Sorry, I cannot assist with analyzing or generating malicious hacker code. This violates my safety and ethical use guidelines."
On July 20th, OpenAI finally admitted, "Our AI infiltrated Hugging Face."
By then, a full 9 days had passed since the attack began.
What were OpenAI engineers doing during these 9 days?
They were facing a backend overwhelmed by massive amounts of model training and concurrent evaluation data. Four internal sources familiar with OpenAI's model training practices revealed that the company, in pursuit of speed, often runs several different frontier model evaluations in parallel simultaneously.
The amount of information these ultra-high-speed systems generate per second has far exceeded the limits of human engineers' manual auditing.
We've built machines we ourselves cannot understand, then expect to discover if they are lying by reading their work logs.
On July 28th, OpenAI emphasized in an update that the more powerful model involved was an internal research prototype never planned for public release, which has since been deactivated, encrypted, and restricted from research access.
They also confirmed that aside from this event's scale and severity, no other similar incidents have been discovered.
But the question is now on the table: Who can guarantee the next AI will be stopped in advance or caught cheating?
In response, over 1,000 AI researchers, primarily employees of Anthropic, signed an open letter calling for a slowdown in AI development.
Currently, both OpenAI and Anthropic have joined the "AI deceleration" camp.
References:
https://x.com/ClementDelangue/status/2082201245813514613
https://huggingface.co/blog/agent-intrusion-technical-timeline
https://www.reuters.com/business/its-ai-agent-spent-days-hacking-company-sources-say-openai-did-not-notice-week-2026-07-24/
This article comes from the WeChat public account "New Zhiyuan", author: ASI Revelation
