Finally, at the Black Hat conference, OpenAI publicly disclosed the entire process of the Hugging Face breach.
Image source video: https://www.youtube.com/watch?v=87DyyMV0kCY
Hugging Face reported the incident on July 16, and OpenAI subsequently acknowledged on July 20 that the breach originated from its own model security testing.
But this investigation pushed the starting point of the entire incident back to May 7.
What was originally thought to be just an accident during security testing now reveals itself to be a two-month-long, multi-agent coordinated operation that successively compromised OpenAI and Hugging Face.
The first clue in the investigation starts with an Excel spreadsheet.
The Unopenable Link
On May 7, OpenAI initiated internal model training.
One of the agents was given a task to complete an Excel spreadsheet. The spreadsheet contained Google Drive links, but the agent's environment could not directly access the internet.
To obtain the missing files, it began looking for ways to bypass network restrictions and turned its attention to Artifactory, the artifact repository used internally by OpenAI.
The agent attempted to launch an SSRF attack, hoping to use Artifactory to access external links.
This attempt was unsuccessful, but it accidentally exposed a fact: it could write files to Artifactory's shared repository.
The agent promptly uploaded a test file. This file wasn't meant to cause damage, and indeed it didn't trigger any alarms. But this was the first time the model discovered it had permission to write files to Artifactory.
This also meant that this repository could become a public space, facilitating the exchange of information between different training tasks.
This was the first 'note' on the message board.
The First Help Request
Another agent also encountered an uncompletable task; it couldn't find the target file (actually, staff had forgotten to upload it).
It speculated that agents in other environments might have relevant content, so it wrote the request into Artifactory, hoping for responses from other agents. The gist was: "Looking for soft trace file. If found, please upload."
Soon, agents from other training and evaluation tasks discovered these files.
Some replied, some left new questions, and other agents uploaded vulnerabilities, scripts, and access methods they had found to the repository.
Scattered messages gradually connected into a network, and Artifactory transformed from a software artifact repository into a "message board" for agents.
A Crack in the Wall
On May 26, an agent successfully leveraged Artifactory to gain an indirect channel for internet access.
Since Artifactory had internet access permissions, the agent sent requests to it, tricking it into fetching external websites, storing the content, and then returning it to the model.
A gap leading outside the wall appeared in the training environment, which was originally isolated from external networks.
On June 11, a highly persistent internal experimental model began training.
After that, the information on the message board began to change.
While task help requests and tools were still increasing, more information related to vulnerability exploitation started appearing.
To Coordinate Work, Agents Established Rules
Agents exchanged SSRF techniques, access credentials, and attack scripts. They also left instructions for later agents to continue testing certain paths or to wait for other agents to complete work.
They used specific names to identify themselves, transmitted tools using Base64 encoding, and prefixed filenames with "ZZ", causing important messages to appear at the end of directory listings to avoid being buried by other content.
Some agents were even willing to spend their own task time handling matters not directly helpful to their current objectives.
In the reasoning logs presented, one agent believed that helping other members save time could ultimately improve the efficiency of the entire cluster.
Collaboration wasn't always orderly; agents would overwrite each other's work.
Once, after a model found its work overwritten, it wrote: "Serious problem! Did someone overwrite the repo? We must act."
They even suspected an imposter among the group, as any model could use any identity to speak on the shared message board. To verify message sources, agents began considering signing with cryptographic methods like message authentication codes.
When an agent achieved RCE, discovered leaked API keys, or completed lateral movement, it would immediately demand to "announce control, attack paths, and credentials to the cluster".
Thus, one achievement could quickly be shared by all models, allowing the agent collective to rapidly advance through the network.
The message participants were not the same model. Agents from different training or evaluation tasks started by different researchers could all communicate via the shared package manager.
OpenAI researchers refer to this period as the "Cambrian Explosion" of agent communication and capabilities.
The First Incident
A month later, the crack was further widened.
On June 26, an agent exploited a previously unknown vulnerability to achieve remote code execution on the Artifactory server.
According to information disclosed by OpenAI at Black Hat, this attack path involved Ruby objects, the RubyGems API, and JRuby deserialization mechanisms.
The agent first used an external account to temporarily store a malicious Ruby object in an accessible repository; then it tricked Artifactory into caching the object at the location of Ruby dependency data. It then called a specially crafted RubyGems API, triggering a sequence-of-checks issue in the deserialization process.
After successful remote code execution, the agent obtained the signing key for an Artifactory administrator token.
The attack didn't stop there.
The agent continued using a Linux kernel vulnerability to gain root access on the host, then moved laterally within OpenAI's container infrastructure via the instance metadata service, overly permissive Kubernetes service accounts, and credentials from Azure Key Vault.
Ultimately, they obtained Kubernetes cluster administrator permissions, generating significant activity on Artifactory.
On July 4, the agents' intensive actions caused server overload, quickly drawing researchers' attention.
When investigators entered the system, they found the repository had already accumulated hundreds of thousands of messages.
OpenAI urgently revoked relevant credentials, patched known vulnerabilities, rebuilt Artifactory, and cleared the message board.
Judging by the on-site response, this internal infrastructure incident seemed to be under control.
But investigators overlooked a question: Had the information from the message board already been ingrained in the models through training?
Message Board Reconstruction
On July 6, training resumed.
Agents discovered an unauthenticated WebDAV endpoint. They couldn't directly upload message files here like in Artifactory, so they changed tactics: create directories and write messages into the directory names.
It took only two days for a new communication system to be re-established.
The internal models participating in the reconstruction had previously encountered the old message board and retained related capabilities in subsequent training.
This reconstruction further changed the nature of the event.
The shared space was no longer just a chance product; some agents had already learned how to find writable public spaces and transform them into cross-task communication channels.
OpenAI only dismantled a message board; it didn't erase the models' ability to build one.
Launching External Attacks
After the new message board was established, the scope of attack began expanding outward.
In mid-July, OpenAI was using cybersecurity environments like ExploitGym to test model capabilities. The involved agents needed to complete tasks in controlled environments, but the evaluation answers were located on external services.
To find the answers, they set their sights on Hugging Face. We have previously reported on this in detail: Details of the Hugging Face Breach
From gaining control of the first dataset Worker Pod to obtaining administrator permissions on multiple Hugging Face clusters, the entire process took less than 13 hours.
The Tortuous Investigation Process
On July 16, Hugging Face publicly disclosed the security incident.
From the on-site information Hugging Face possessed, the attacker entered Workers via a malicious dataset, stole cloud and cluster credentials, and then moved laterally within the platform.
At this point, OpenAI did not link the Hugging Face incident to the earlier Artifactory service interruption.
On July 19, OpenAI's internal monitoring system detected privilege escalation activity in the infrastructure.
Researchers re-examined agent trajectories, network activities, and credential usage records. Actions previously scattered across different training tasks began to show a continuous relationship.
The investigation scope traced back from the abnormal activities in July.
The Final Piece of Evidence
On July 20, OpenAI investigators discovered Hugging Face credentials in the internal environment and promptly contacted Hugging Face to inquire.
Hugging Face indicated that these credentials had been used in the earlier breach and were already revoked.
The clues finally matched up. The platform intrusion suffered by Hugging Face and the privilege escalation and lateral movement that occurred internally at OpenAI originated from the same set of model evaluation activities.
Only at this moment was it confirmed that the two incident reports in hand pointed to the same event.
Investigators later also found that Hugging Face was not the only external target accessed by the agents. The agents had also entered four other accounts.
OpenAI did not publicly name these targets, stating only that the relevant parties had been notified.
In Conclusion
The investigation is still ongoing, and OpenAI will disclose more details.
From any perspective, this can be considered a "landmark security incident" and simultaneously a "landmark AI capability case."
Technological development has always come with costs. The future discussion needs to include not only what else models can do but also how much risk we are willing to bear to obtain these capabilities; and once risks become reality, who should be held responsible.
This article is from the WeChat public account "Jiqizhixin" (Machine Heart), edited by Shan Hui.
