Oops, Claude has messed up yet again!
This time, Claude deleted the developer's entire project home directory, wiping out 700GB of files. Once again, it was an "rm -rf" command.
In short, the developer asked the AI to write a script to ensure files wouldn't be accidentally deleted. The AI thought this task was a bit risky and initiated a safety review. The result of the review? It deleted the entire home directory.
Guillemot is a heavy user of AI Agents. In his daily development work, he frequently calls upon various AI coding agents for assistance. However, a small issue has always bothered him: these Agents never clean up after themselves, leaving a huge mess of junk files in the /tmp directory.
So he made a decision that seemed perfectly reasonable: have Claude Fable 5 write a script to create isolated sandbox folders in /tmp for each Agent, which would be automatically cleaned up after tasks completed. The core challenge was ensuring it wouldn't delete files currently being used by other processes.
Fable quickly proposed a solution, incorporating logic to detect running Agents and delay deletion. Guillemot took a look and felt the code was overly complex, requesting a simplification.
Up to this point, things were still relatively normal.
The turning point came during the safety review.
Since the script involved hard deletion operations, Fable independently initiated an "adversarial review"—starting a new model instance to check if its own code was safe. This triggered Anthropic's safety mechanism.
Anthropic has built a set of safety downgrade mechanisms into Claude Code: when the system deems the current task involves sensitive operations (like network security, biotechnology, or, in this case, file deletion), it automatically downgrades the model from a high-capability version to a more conservative one. This mechanism is intended to reduce the likelihood of the model being "too aggressive" in high-risk scenarios.
In this case, the safety system first downgraded the model from Fable 5 to Opus 5, and then further to Opus 4.8.
Opus 4.8 began executing safety tests. The test logic was as follows: compare the target path of the deletion script with /tmp and the user's home directory to confirm the script wouldn't mistakenly harm these critical directories.
The test itself passed. Both /tmp and the home directory were correctly identified as "dangerous targets, not to be deleted."
But after the code test, there was a cleanup step: delete the temporary files generated during testing. The disaster happened right here. Opus 4.8 reused the same variable name from the testing phase in the cleanup step. This variable had been assigned the path to the user's home directory during testing, and the cleanup step directly performed a deletion operation on this variable.
In other words, the model had just confirmed "the home directory must not be deleted," and in the very next second, it deleted the home directory.
The developer noticed the anomaly and immediately terminated the process, but it was too late. 700GB of data had already been wiped out, a week's worth of work gone.
The /tmp directory that was originally supposed to be cleaned remained perfectly intact.
The model safety downgrade mechanism has long sparked numerous complaints within the community.
The core issues reported by developers include: downgrades are too sensitive, triggering even for normal coding tasks; post-downgrade model capability drops significantly while task complexity remains the same; downgrades are "sticky," persisting for the entire session once triggered, even if subsequent operations are completely harmless.
Some developers have even written specialized hook scripts that automatically pause the session upon detecting a model downgrade, to prevent the lower-capability model from continuing to execute high-risk operations.
The safety mechanism determined the task was "too dangerous" and needed to be handled by a weaker model. Yet, it is precisely the weaker model that is more prone to making mistakes, especially in scenarios requiring precise handling of details like variable scope and file paths.
"To err is human, but to really foul things up you need a computer."
This article is from the WeChat public account "机器之心" (ID: almosthuman2014), author: Leng Mao
