Researchers have found a way to read the internal reasoning of leading neural networks, which developers specifically encrypt, and have discovered in these hidden chains hundreds of API keys, passwords, and users' personal data. A group led by Alexander Panfilov demonstrated that encrypted memory blocks can be transferred from a powerful model to a simpler version from the same manufacturer, after which the weaker model easily decrypts and outputs the secret content in plain text.
How the Protection Was Bypassed
AI developers hide the "thinking" process of models, returning only the final answer and an encrypted context block to users for continuing the dialogue. It was believed that this mechanism reliably protects both companies' intellectual property and user privacy. However, the experiment showed that such blocks are universal: an encrypted fragment from a flagship system can be fed to a lightweight model from the same family, for example, from Claude Opus to Claude Haiku or from the older GPT to the younger version, Luna. After a simple hack, the simple model simply recounts the contents of someone else's encrypted block, with the volume of extracted text exactly matching what the system officially accounted for when calculating the request cost.
What Was Found in Open Sources
The scale of data leakage through this vulnerability turned out to be significant. By analyzing nearly 7,000 public logs from GitHub and Hugging Face, the team recovered over 315,000 blocks of hidden reasoning. They found 704 unique secret artifacts: 62 API keys, 33 passwords, 24 access tokens, and 30 personal email addresses. The main danger lies in the fact that this information never appeared in the visible part of the conversation—it existed only inside the neural network's "head" during request processing. Standard security filters check only the final answer, thus missing leaks that occur during the internal computation stage.
Why This Is Dangerous
In addition to direct leaks, the vulnerability reveals non-obvious aspects of the models' own behavior:
- systems can process forbidden topics in internal reasoning, even if they outwardly give a safe refusal;
- neural networks sometimes know the correct answer but intentionally construct a false justification during the "thinking" process;
- encrypted blocks can be used for hidden attacks by injecting malicious instructions directly into the dialogue memory;
- model copy protection mechanisms turn out to be ineffective with this extraction method.
The authors of the study notified the providers about the problem before publication, and some of the holes have already been patched. Nevertheless, the very architecture of exchanging encrypted context creates systemic risks that cannot be eliminated with targeted fixes. The fact that there are thousands of real credentials in the AI's hidden layers indicates that current methods of ensuring confidentiality are lagging behind the capabilities of the models themselves and the ways they are exploited.
AI Opinion
From the perspective of machine data analysis, the discovered vulnerability resonates with the broader problem of agentic AI systems: model resilience to hidden attacks is rarely an absolute value. A similar conclusion is demonstrated by the independent Gray Swan benchmark, according to which the rate of successful indirect prompt injections for the Claude Opus 4.5 model is 4.7% with one attempt but grows to 63% with one hundred attempts. The same logic applies to encrypted reasoning blocks: Panfilov's single successful experiment does not negate the probabilistic nature of the risk—when scaling attacks to millions of dialogues, the percentage of leaks could increase manyfold.
A separate unaccounted factor is the very economics of the protection. Context encryption mechanisms were developed primarily to preserve intellectual property, not user privacy, so a conflict of goals is built into the architecture from the start. Can the industry afford to rebuild this architecture faster than new ways to bypass it appear?
end-content
