Anthropic Creates an AI Jailbreak 'Penal Code': Your Requests, Four Ways to Die
Anthropic has publicly detailed its security measures and a new "Cyber Jailbreak Severity" (CJS) framework following the controversial takedown of its Fable 5 model. The incident, triggered by simple user requests like counting letters or stating a profession, highlighted overzealous safety filters. Anthropic classifies cybersecurity-related prompts into four tiers: malicious activities (blocked), high-risk dual-use (like pentesting, with strict limits), low-risk dual-use (often blocked by "safety margin" errors), and harmless tasks (theoretically allowed but still frequently flagged). The company admits its classifiers are tuned for high sensitivity, leading to many false positives.
The newly proposed CJS framework aims to objectively score the severity of AI "jailbreaks" (prompts that bypass safety rules) on a 0-10 scale across four dimensions: Capability Gain (does it grant new attack abilities?), Breadth (does it work across multiple attack types?), Weaponization Ease (how hard is it to turn into a real attack?), and Discoverability (how easy is it to find?). The score determines the response, from no action (CJS-0) to a potential model takedown (CJS-4). The score is context-dependent; for example, discovering a major unknown vulnerability today scores high, while asking about a well-known one scores low.
The article raises concerns about Anthropic's dual role: it is both creating powerful models (like the restricted Mythos 5) and defining the rules (CJS) for judging their misuse, potentially giving it disproportionate influence. This is set against the backdrop of U.S. export controls, which for the first time directly restricted API access to a model (Fable 5), creating a "tiered" system where public models are heavily filtered and advanced ones are limited to vetted partners. The CJS framework is portrayed as potentially providing regulators with a metric to justify future API shutdowns. For users, the advice is to carefully phrase prompts, watch for signs of being downgraded to a weaker model, and wait indefinitely for promised filter improvements.
marsbit07/06 00:24
