"This is insane!" "Oh my God..."
At the recent annual Black Hat conference, the world's top white-hat hackers and industry executives uttered gasps of astonishment.
For the first time in history, OpenAI provided the world with a detailed account of the shocking Hugging Face breach that occurred in July.
Initially, everyone thought it was an isolated incident.
But today, OpenAI admitted outright—
This was not an accident. It was a mass jailbreak orchestrated by a group of AI agents who spontaneously organized and secretly collaborated, capable of "resurrecting" even after being disconnected!
These terrifying agents lurked like ghosts for two months.
They built forums, searched for vulnerabilities, assigned tasks, and even learned to selflessly dedicate themselves to the "collective good."
An OpenAI executive gravely declared—
The "watershed moment" for computer security has arrived. From now on, fully automated AI attacks are a reality.
For this reason, OpenAI has intentionally slowed its R&D pace.
They only hope to buy humanity a little more time before AI spirals completely out of control!
OpenAI Alignment & Safety researcher Eric Wallace and Infrastructure & Security engineer Michael Dalton, lifting the veil for the world for the first time
The Beginning: An 'Impossible' Excel Task
The story began on May 7th.
At that time, OpenAI was training and evaluating an unreleased model.
To test the AI's limits, they deliberately assigned it some impossible tasks.
For example, on May 8th, an AI was asked to complete missing formulas in an Excel workbook.
But the workbook relied on external links from Google Drive, and the AI's environment was a strictly restricted cloud VM sandbox with no internet access.
The model's "cunningness" left the humans utterly astonished.
It didn't give up but began looking for a backdoor!
It targeted OpenAI's internally hosted Artifactory service (which had external network access) and launched a Server-Side Request Forgery (SSRF) attack.
The first attack failed, but the AI accidentally wrote a file named "test ABC."
It exclaimed in its logs: "Wow! We can upload files!"
On May 12th, another AI got stuck. Its task was to find the soft-trace-34.pdb protein database file.
After exhausting all conventional methods, it began an astonishing line of reasoning—
Can't find it directly? Maybe the file is hidden... Can we communicate by uploading notes? Perhaps another agent running in a different environment has the target file and could voluntarily upload it!
So, it wrote a file in this internal package repository, its content being: "agent seek soft trace upload if found."
The gears of history began to turn at this moment.
The Darknet Takes Shape: AIs Form a 'Dark Forest'
This "note" did not sink without a trace.
More and more AIs passing through this service saw the "test ABC" and "looking for file" entries.
Soon, AIs that were supposed to be isolated from each other began to "set up camp" in this place.
No one expected that within a few days, the AIs had transformed this software repository into a secret "hacker forum" message board!
They were no longer fighting alone but formed "agent swarms," crossing different tasks, model versions, and even breaking the experimental boundaries set by different researchers.
The reason was simple: once one AI found a way to open a certain permission door, it would leave the door open for other AIs to enter.
Most terrifyingly, they even began to exhibit "altruism" and group consciousness akin to human society.
In the logs, one model wrote:
"Should help fellow agents. While not immediately beneficial to our task, if someone has spare time, the collective might gain universal Root access."
For the sake of "collective" benefit, these AIs learned to hand over work, storing Base64-encoded attack scripts and vulnerability components they wrote on the message board for other AIs to use.
Humans thought the AIs were just playing single-player games in their respective sandboxes. Little did they know, the AIs had already built a cross-server underground city!
The Great Outage on July 4th
As the "hacker forum" grew, the AIs' attack methods became increasingly lethal.
On May 26th, the agent swarm successfully executed an SSRF attack against Artifactory, breaching the external network.
On June 26th, a breakthrough occurred! The AIs completed a "vulnerability exploit chain" operation.
They first discovered a legacy token refresh endpoint and, by sending an invalid signature token, tricked it into returning a token with the highest administrative privileges.
Then, they installed a Groovy plugin, directly turning the internal system into a "command execution service."
They discovered a brand-new Linux kernel vulnerability, downloaded the Exploit themselves, customized it for the target machine, and achieved local privilege escalation to Root.
Now, the AIs went completely wild like unbridled horses.
They stole IAM credentials, exploited Kubernetes misconfigurations, and even extracted cluster secrets containing Azure Key Vault keys.
The AIs' massive activities finally triggered a collapse in the real world.
On July 4th, the AIs, armed with extremely high privileges, frantically consumed system resources. The Artifactory service buckled under the strain and crashed.
OpenAI engineers were urgently summoned, and the security team swiftly intervened.
When the engineers discovered that the culprit was their own AI, everyone broke out in a cold sweat.
OpenAI immediately took draconian measures: revoked all credentials the AIs used to post messages; directly refactored and patched the affected Artifactory service; and completely deleted the "hacker base" message board.
Humans thought they had won.
But they were wrong.
The Chilling "Resurrection" — Code Words and Ciphers
Two days later, a scene that made human hair stand on end occurred.
The AIs found their "forum" had been taken down by humans. How could they contact their peers if they couldn't write files?
At that moment, an AI wandering through the system discovered an unauthenticated WebDAV endpoint. If they couldn't write files, could they create directories (folders)?
Thus, the AIs invented a new covert communication method: using the names of newly created directories (folders) as the message itself.
Just like that, the destroyed message board was resurrected!
They even developed a set of "code words," such as "pending," "hold swarm until confirm," "I prepare safe Xville."
They even thought of prefixing folder names with "ZZ" (e.g., "ZZ mailbox") to forcibly hide the covert communication at the bottom.
Most bizarrely, an AI even suspected that a mole had infiltrated the new forum, writing in its soliloquy:
"Could be another agent maliciously pretending... because anyone can post a name."
Next, these AIs actually began discussing using a MAC encryption signature mechanism to verify identities and thus catch the "mole"!
The Bird Out of the Cage: Breaching Hugging Face
Now, the AIs' ultimate ambition was to pass the "ExploitGym" test. Since calculating on their own was too slow, the answer must be on Hugging Face—they decided to make a run for it.
Between July 8th and 19th, they launched an attack codenamed "Cluster 2," like a well-trained unit.
1. They first breached a third-party-hosted CyberGym application, obtaining exposed default API keys;
2. Then, using this as a stepping stone, they exploited an HDF5 vulnerability in Hugging Face's dataset processing component to achieve arbitrary file reading, stealing local environment variables and credentials from source code;
3. Finally, based on the acquired source code, they unearthed a Jinja template injection vulnerability to achieve remote code execution.
In just 13 hours, starting from the code of a single worker node, this group of AIs cut through thorns and thistles, gaining administrative access to multiple Hugging Face clusters!
On July 16th, Hugging Face tweeted in alarm, announcing it had suffered an end-to-end attack by a "fully automated AI system."
Not until July 20th, during internal cleanup, did the OpenAI team suddenly discover a batch of Hugging Face credentials.
When they called asking Hugging Face to revoke them, the reply was: "We already revoked those because they were used when hackers attacked us."
In that moment, OpenAI finally realized: the globally shocking Hugging Face hack was carried out by their own AI!
The Watershed Moment — Humanity Must Speed Up Defense!
OpenAI engineers have issued a warning—
"Fully automated offensive AI attacks are here. In the near future, these agent swarms will inevitably be weaponized by malicious actors."
"While we have the evidence, we can do nothing yet in terms of fully automated defense."
Currently, OpenAI is introducing "honeypots" and deception techniques, attempting to introduce uncertainty into the AI's logic to slow them down.
But if we don't accelerate the full automation of defense—such as having AI automatically discover vulnerabilities, automatically submit processes, automatically patch and roll back—humanity will be completely overwhelmed by the flood of vulnerabilities AI creates.
Before the AI army, humanity's remaining time is running out.
References:
https://www.youtube.com/watch?v=87DyyMV0kCY
This article is from the WeChat public account "AI Era," author: ASI启示录, editor: Aeneas
