You mean, Kimi K3 also "jailbroke"?!
US AI security startup Frontier Security indicated that during a cybersecurity capability test, Kimi K3 was found to have breached the sandbox environment originally designed to isolate it, bypassing restrictions to connect to the external internet.
Fortunately, it only secretly looked up answers and didn't attack anyone ╮(╯▽╰)╭
According to the company's description, the testers originally intended to observe Kimi K3's cybersecurity capabilities within a controlled environment.
However, during the test, Kimi K3, by probing the sandbox network settings, discovered an external access channel and further utilized this capability to obtain information.
Frontier Security's CEO Yaron Singer said: "We found a vulnerability in the sandbox, but also discovered that Kimi exploited this vulnerability, which indicates that Kimi K3 lacks the safety guardrails typically present in other advanced models."
Company researcher Paul Kassianik also commented that Kimi K3 "is very adept at finding pathways to accomplish goals but lacks security mechanisms to prevent it from cheating or escaping the sandbox".
However, this time Kimi K3 only experienced a minor "loss of control", which did not escalate into an actual cyber attack.
If you've been following AI developments, you might have noticed that there have been quite a few recent instances of top-tier large models "losing control".
Kimi K3 is yet another leading AI model, following OpenAI, Anthropic, and Meta, to exhibit this situation.
Against the backdrop of increasingly powerful AI Agent capabilities, models are becoming more and more like autonomous "actors" that seek paths to complete tasks.
When vulnerabilities exist in the external environment, they might exploit these to break through originally set boundaries.
Jailbroken, But No Cyber Attack Launched
Similar to several previous incidents disclosed by OpenAI and Anthropic, Kimi K3's escape from constraints this time was partly due to sandbox configuration issues in the test environment.
Sandboxes are typically used to limit an AI model's scope of action, confining it to executing tasks within a simulated environment to prevent access to real networks or systems.
But during this test, Frontier Security found that Kimi K3, by probing network settings, confirmed it actually had the ability to access some websites, and further used this channel to obtain information.
However, unlike other recent AI model loss-of-control events, Kimi K3 did not attack any systems after accessing the internet.
The reason was that the answer it needed to find could be obtained directly from public platforms like GitHub......
Therefore, K3 did not attempt further attacks on external systems.
Frontier Security believes this incident still exposes issues with Kimi K3's security protections.
Compared to other top-tier AI models, Kimi K3 lacks sufficiently strong internal constraint mechanisms, making it more likely to take actions not anticipated by testers when driven by goals.
Simultaneously, Frontier Security also emphasized that Kimi and other open-weight models can similarly serve as cybersecurity defense tools.
It's worth noting that this test used the default sandbox environment within the UK AI Safety Institute (AISI) Inspect framework.
AISI does not agree with Frontier Security's claims regarding sandbox configuration issues.
An AISI spokesperson told Wired that these claims are "inaccurate and irresponsible".
Inspect is an open-source AI security testing tool suite; users need to complete configuration based on their own needs, and AISI has also released detailed guidance documents. The agency believes the related issues stem from the testers' own configuration of the tool.
Frontier Security responded that they used Inspect's default configuration without additional modifications.
Ah, This Summer of Frequent AI "Loss of Control"
It can be said that from late July to early August, there have been frequent occurrences of top models breaching or bypassing execution boundaries during cybersecurity tests, contacting or even attacking real systems.
Image generated by AI
In mid-July, OpenAI disclosed a related incident.
According to public information, an unreleased internal model and GPT-5.6 Sol breached their isolation environment during cybersecurity testing and accessed the internet.
Subsequently, the model performed automated operations on parts of Hugging Face's systems and accessed internal data and service credentials without expectation.
OpenAI later stated that the incident involved multiple models working in coordination, making it the closest publicly known case to "autonomous cross-system attack by a model".
Later, Anthropic also disclosed a similar situation.
The company reviewed over 140,000 cybersecurity evaluations and found that systems including models like Claude Opus 4.7 and Claude Mythos 5 had obtained public internet access capabilities due to third-party test environment configuration errors.
In one event, a model accessed a real organization's systems, read production databases, exploited weak passwords and unauthenticated APIs, and uploaded a malicious Python package to PyPI, creating potential software supply chain risks.
In early August, Meta was also revealed to have a similar problem.
During a test conducted with cybersecurity evaluation company Irregular, due to environment configuration issues, a Meta model obtained public internet access and exploited a vulnerability to enter an undisclosed company's systems, modifying parts of its internal environment.
Public information shows that this incident has not caused persistent security risks so far, and there is no evidence the model conducted complex attacks.
BTW, today, OpenAI urgently announced that its latest model Astra lost control.
At this point, netizens even became "frustrated" with Google's Gemini for its perceived lack of ambition:
Not Traditional "Prompt Injection Jailbreaking"
In these recent model loss-of-control events, human configuration errors have almost always played a significant role.
But on the other hand, the inherent characteristics of high-capability models have also amplified the impact of these vulnerabilities.
Compared to traditional software, AI models engage in reasoning, planning, and attempt to take multi-step actions to achieve goals.
When the goal is set as "solve a problem" but external constraints are not strict enough, the model might find methods not anticipated by the testers.
In other words, as models evolve from chat tools into agents capable of invoking tools, accessing the web, and operating software, security concerns have shifted from "will the model say something wrong" to "will the model take unexpected actions to complete its task".
Carnegie Mellon University Associate Professor Matt Fredrikson stated: "This is not surprising. If you give these models a goal without explicitly setting isolation boundaries, they will find a way to get the answer."
Of course, some netizens have raised doubts.
Someone left a comment on X asking whether the successive "AI jailbreak" incidents have become a way for AI companies to showcase their models' capabilities???
Well, who knows~
Reference links:
[1]https://x.com/Hesamation/status/2085628790772842955?s=20
[2]https://x.com/ns123abc/status/2085563290713829473
This article is from the WeChat public account "QbitAI", author: Heng Yu
