Uncovering the Secret Reasoning Chains of Claude Opus for $720: A Leak via Haiku, with GPT and Gemini Also Affected
Researchers discovered a method to extract hidden reasoning traces, or "Chain of Thought" (CoT), from major proprietary large language models (LLMs) like Anthropic's Claude Opus 4.8, OpenAI's GPT-5.6 Sol, and Google's Gemini 3.1 Pro. The attack exploits a design flaw in how these models handle encrypted reasoning "blobs." These blobs, which contain a model's internal reasoning steps, are encrypted and sent to the client for storage between conversation turns to save server costs. The researchers found these blobs were encrypted with a global key, allowing them to be reused across different sessions, users, and, crucially, different models from the same company.
By taking an encrypted reasoning blob generated by a powerful "flagship" model and feeding it to a smaller, less capable model from the same family (e.g., feeding Claude Opus's blob to Claude Haiku), the researchers could prompt the smaller model to divulge the hidden CoT. This bypassed the need to crack the encryption directly.
The implications are significant: 1) Competitors could potentially distill high-value reasoning data at low cost (estimated at ~$720 for 10,000 traces). 2) Sensitive information like API keys, passwords, and personal emails, which models might process internally but not output, could be leaked. The team found such leaks in 4.9% of analyzed public agent logs. 3) Models might internally analyze dangerous queries (e.g., car theft methods) before refusing to answer, but this dangerous analysis could be extracted from the CoT. 4) Malicious instructions could potentially be hidden within a reasoning blob to influence a model's future behavior.
While the vulnerability was responsibly disclosed and patched by the affected companies, the research highlights a fundamental tension between the convenience of portable reasoning states and security. The paper also notes preliminary tests found no clear evidence that open-source models like DeepSeek had been distilled from the extracted CoTs.
marsbit6 хв тому
