Anthropic Admits Fault, Evidence of Secretly Downgrading Claude's Intelligence

marsbitPublicado a 2026-08-24Actualizado a 2026-08-24

Resumen

Anthropic publicly apologized after users discovered a hidden A/B test in Claude Code (versions 2.1.237+) that seemingly "dumbed down" the model. Developer argofowl found that selecting the "high" reasoning effort was being mapped internally to a value of "10," which previously corresponded to the "low" setting. While Anthropic engineers clarified this was only a change to the numerical mapping scale and assured it did not affect performance based on internal evaluations, the lack of communication about the test sparked widespread user backlash and self-tests on social media. Separately, significant complaints emerged about the flagship Opus 5 model, with users reporting it became "unstable," prone to lazy outputs, factual errors, and repetitive corrections. An Anthropic engineer acknowledged Opus 5's performance instability, calling its resolution a top priority. The incident highlights a growing industry-wide rift between improving benchmark scores and declining user-perceived quality and reliability. It underscores the lack of transparency in continuous model updates, where server-side changes, A/B tests, and configuration adjustments occur without clear user notification, eroding trust as AI models become critical infrastructure.

Claude became dumb overnight, the whole internet is debating it!

Just yesterday, developer argofowl spent a whole afternoon, almost turning Claude Code inside out.

He troubleshooted step by step, first suspecting t3 code crashed, then thinking his own code had a bug.

By the end, he even started wondering—had he broken his Mac?

When argofowl finally opened the API's real request logs, the truth became clear. It clearly showed the number "10".

However, in the Claude Code backend, he had clearly selected "high", the highest reasoning level.

Who would have thought that Claude Code's update logs contained not a single word about this.

Reasoning "high" becomes "10"

Claude Code "Dumbening" Exposed

argofowl discovered that starting from Claude Code version 2.1.237, the model reads the "high" reasoning level as 10 out of 100.

And this number was exactly the value previously corresponding to the "low" level.

Looking deeper, it was found that Anthropic included Fable 5 sessions for Claude Code version 2.1.236 and above into an experiment called "compressing the effort numerical scale".

However, older versions and Opus 5 are not affected.

This is most likely an A/B test, so not everyone will encounter it.

For developers, this is the real pain point. A model being slightly stronger or weaker is tolerable.

But pulling me into the experimental group without planning to tell me: What exactly am I debugging all afternoon? Am I debugging my own code, or your A/B test?

Unexpectedly, after tech blogger Chubby reposted this, the AI circle exploded—

It looks like Anthropic quietly made the model dumber, without telling anyone.

Instantly, self-test posts titled "Did Claude get dumber?" flooded X (formerly Twitter).

Some posted output comparisons of the same prompt across different versions, others dug out their conversation records from two weeks ago for line-by-line diffs.

Anthropic Admits Fault, Engineers Step In

Facing this storm, the response from Claude Code engineer Thariq Shihipar came quickly.

We sometimes test API service configurations in Claude Code first, before deciding whether to roll them out fully.

The experiment currently running only changed the way the effort numerical values are mapped. So some people will see Claude saying it's "10".

The key point is, this scale is not 0 to 100. That number alone has no meaning. The effort you selected is the effort you get.

He emphasized that the team conducted in-depth evaluations to confirm this does not affect model performance.

Opus 5 Major Intelligence Downgrade, It's True

Just after resolving the Fable issue, Chubby bluntly stated that Opus 5 now feels like a significant downgrade.

It's always perfunctory, frequently making basic mistakes. Once pointed out for not following instructions, it just mechanically replies with that line—

You are right, I overlooked that. Over and over again, endlessly.

In fact, a few days ago, people had already noticed the obvious "dumbening" problem with Opus 5.

Besides the issues mentioned above, it would create bugs, then spend a lot of time fixing them, repeatedly self-correcting within the same task.......

After being questioned by netizens, engineer Thariq publicly admitted—Opus 5 is a model with "very unstable performance", fluctuating up and down, inconsistent.

The team is working hard internally to resolve this issue. It's the highest priority for us.

Benchmark Scores Always Up

User Experience Consistently Down

This Opus 5 storm actually exposes the most awkward fissure in the entire industry:

Benchmark scores and user experience are systematically decoupling.

On one side, there's a dazzling, almost flawless report card: Composite score 82.72, SWE-bench Pro 79.2%, Terminal-Bench 86.7%.

On the other side, there's the starkly opposite real user feeling: "Long-winded", "lazy", "loves to argue".

The most absurd part is that both kinds of evaluations coexist on Opus 5.

Moreover, "the model getting dumber" is not a problem unique to Anthropic.

Now, large model version updates are becoming the most opaque black box in the entire AI industry.

Traditional software has semantic versioning, change logs, and rollback mechanisms. Developers can clearly know which version they are using and what changed.

Large models are different.

Under the same model name, the server side may conduct A/B tests at any time, change quantization schemes, adjust model routing, or even alter inference resources.

The only dashboard people have left is their own subjective experience.

Yet intuition is the most easily dismissed, and hardest thing to disprove.

The greatest value of this storm is that it has brought a long-standing, hidden wound in the industry completely into the open:

When models become infrastructure, stability is a trust contract. Benchmarks can be used for marketing, stability can only be earned by delivering on promises, time and time again.

References:

https://x.com/trq212/status/2091252347913773169?s=20

https://x.com/kimmonismus/status/2091178321669198014

This article is from the WeChat public account "新智元" (New Wisdom Era), author: ASI启示录, editor: 桃子 (Peach)

Preguntas relacionadas

QWhat did the developer argofowl discover when investigating Claude Code's performance?

AHe discovered that from Claude Code version 2.1.237, when the 'high' reasoning effort was selected in the interface, the model was actually receiving a value of '10', which was the value previously associated with the 'low' effort setting.

QHow did Anthropic engineer Thariq Shihipar explain the '10' value appearing in API logs?

AHe explained that it was part of an experiment on a 'compressed effort value scale'. He stated that the scale was not 0 to 100, and the number itself had no standalone meaning. The effort level selected by the user was the effort level they received, and internal evaluations confirmed it did not affect model performance.

QWhat were the user-reported issues with Claude Opus 5 mentioned in the article?

AUsers reported that Opus 5 felt like a significant downgrade. It was described as being perfunctory, making frequent low-level errors, and when its mistakes were pointed out, it would mechanically repeat apologies like 'You're right, I overlooked that' without substantive correction.

QWhat was engineer Thariq Shihipar's admission regarding Opus 5's performance?

AHe publicly admitted that Opus 5 was a 'highly variable' model, meaning its performance was unstable and inconsistent, and that resolving this issue was the team's highest priority.

QAccording to the article, what major industry problem does the Claude incident highlight?

AIt highlights a systemic disconnect between benchmark scores and user experience. It also exposes the lack of transparency in large language model updates, where changes like A/B tests, quantization adjustments, and inference resource allocation can happen server-side without clear user notification, making user perception the only unreliable dashboard.

Lecturas Relacionadas

Hyperliquid's Compliance Journey: From Permissionless to Permissioned via HIP-3

Hyperliquid’s Compliance Path: From Permissionless to Permissioned HIP-3 Hyperliquid currently blocks U.S. access because its permissionless, on-chain infrastructure conflicts with U.S. market structure laws, which restrict futures trading to registered exchanges, clearinghouses, and brokers. Through its Hyperliquid Policy Center (HPC), the project is advocating for regulatory modernization, proposing that regulated entities be allowed to build products on HyperCore (its exchange and clearing layer) while fulfilling their compliance obligations. The platform’s modular stack separates roles like a traditional exchange (DCM), clearinghouse (DCO), and broker (FCM), but reconstructs them on-chain with code. This enables permissionless access, self-custody, and 24/7 global trading, but clashes with U.S. rules requiring KYC, specific margin models, and custodial arrangements. To resolve this, HPC is engaging with U.S. regulators (CFTC, SEC) to seek clarity that deploying on-chain software does not itself trigger licensing, and to establish exemptions allowing non-custodial wallets to route users to regulated derivatives. Recent political signals suggest openness to this approach. On the technical side, Hyperliquid Labs has introduced permissioned HIP-3 deployers on testnet. These allow regulated entities to launch markets, perform KYC, and whitelist compliant users. While these create separate order books, whitelisted market makers can bridge liquidity between them, ensuring deep, shared liquidity across the same L1. Features like payload-based “PA” permissions enable DEX-level account controls (e.g., reduce-only orders), mirroring traditional broker authorities. The strategy is not to open the native, permissionless front-end to U.S. users, but to position Hyperliquid as neutral infrastructure that U.S. regulated firms can use while meeting their legal duties. This paves a compliant path for U.S. investor access while preserving the protocol’s core, permissionless nature.

marsbitHace 14 min(s)

Hyperliquid's Compliance Journey: From Permissionless to Permissioned via HIP-3

marsbitHace 14 min(s)

Two Funding Rounds in Three Months: The Chinese Version of Palantir is on Fire

Investment Community AI has learned that Beijing Zhongshu Ruizhi Technology Co., Ltd., a domestic industrial-grade causal intelligence and high-reliability decision-making AI company, has recently completed a strategic financing round worth hundreds of millions of RMB. This round saw participation from China Internet Investment Fund, Suzhou Chuangtou National Social Security Fund, Financial Street Capital, ICBC Capital, Kunlun Capital, among others, with existing shareholders also increasing their investment. This follows a Series B funding round in the hundreds of millions completed just three months prior. The rapid succession of two major funding rounds signifies strong market recognition of the company's underlying original technology and scaled commercial implementation. Often referred to as the "Chinese version of Palantir," Zhongshu Ruizhi is entering a new phase of accelerated technological iteration, widespread scenario replication, and scaled performance release, mirroring the explosive growth of China's AI market. Founded in April 2020 by Dr. Han Han, a Tsinghua University Ph.D. and former core drafter of national AI policies, the company is mission-driven to "move AI from the digital world to the physical world." It focuses on the high-reliability, strong-decision industrial AI track and enterprise-grade AI Agent full-stack infrastructure. The team tackles the challenge of applying AI to China's vast and complex industrial and energy systems by developing a new intelligent operating system from scratch. Its core technological breakthrough lies in three proprietary底层 technologies: meta-causal cognitive theory, causal models, and a dynamic ontology engine. These address critical pain points of generative large models in industrial settings—such as AI hallucinations, insufficient reasoning, lack of temporal logic, unverifiable decisions, and multi-source rule conflicts—thereby providing trustworthy, explainable, and executable智能决策 capabilities. Commercially, Zhongshu Ruizhi has achieved scaled deployment, serving over 50 central state-owned enterprises and industrial groups in sectors like power, petroleum, and aerospace, with implementations in more than 800 highly complex production scenarios. The company reported doubled revenue in 2025, demonstrating strong self-sufficiency and a viable business model—a rarity among new-generation AI firms. The latest funds will be allocated towards advancing foundational theoretical research, replicating successful application models to expand market presence (including overseas), and attracting top-tier talent. Lead investor China Internet Investment Fund highlighted that in the current shift from general AI capability contests to deep industrial empowerment, industrial-grade causal intelligence is crucial for building China's modern digital foundation and fostering new quality productive forces. They expressed support for the company's efforts to define decision-making paradigms and trustworthy standards for industrial intelligence, aiming to secure a rule-making voice in the global physical AI arena.

marsbitHace 25 min(s)

Two Funding Rounds in Three Months: The Chinese Version of Palantir is on Fire

marsbitHace 25 min(s)

The Biggest Political Economy Question in the AI Era: As Robots Become More Capable, How Do Humans Share the Value?

In the AI era, the most pressing political economy question is: as machines become increasingly capable, how can humanity share in the value they create? An article originally critiquing China's tech focus has sparked a deeper debate on this global challenge. Historically, industrial progress improved efficiency but still relied on human labor for wealth creation and distribution. AI is fundamentally different—it is now replacing cognitive and knowledge work. As AI and robots take over more tasks, economic growth may continue while direct human participation in value creation shrinks, creating a core tension between productivity gains and widespread income generation. The issue is not unique to China. While leading tech companies amass enormous wealth, labor's share of income is declining globally. The core problem is a broken link: technological innovation and corporate profits are not translating into sufficient consumer income and demand. Three potential paths forward are outlined: a traditional capitalist model where profits primarily go to capital owners; a state-capitalist approach with public investment in AI; and more innovative models like digital sovereign wealth funds, universal shareholding, or AI-era basic income schemes to directly distribute AI-generated value. The future competitive advantage may lie not just in technological supremacy, but in which society can build a new, inclusive distribution system for the intelligent economy. The ultimate challenge is ensuring that as AI creates value, humans have a means to obtain income and share in the resulting widespread social benefits.

marsbitHace 35 min(s)

The Biggest Political Economy Question in the AI Era: As Robots Become More Capable, How Do Humans Share the Value?

marsbitHace 35 min(s)

Generating Profits for Seven Consecutive Quarters, Emerging Markets Carry Trade Outperforms Everything

For the seventh consecutive quarter, dollar-funded emerging market carry trades have delivered positive returns, marking the longest winning streak since 2008. According to Bloomberg's index, this strategy has gained approximately 22% since late 2024, outperforming U.S. Treasuries, emerging market sovereign, and corporate dollar debt. The core of the trade involves borrowing low-interest currencies like the U.S. dollar, euro, or yen to invest in high-yielding emerging market assets, such as Turkish lira bonds offering over 40% returns. Returns were amplified by favorable currency moves, with the dollar weakening against most emerging market currencies and other traditional funding currencies. For instance, the trade gained 48% on the Colombian peso in the past year. A key test came in August 2024 with a historic joint U.S.-Japan currency intervention, which caused only a modest 1% dip in the carry trade risk premium as investors shifted funding from the yen to the euro and Swiss franc. Looking ahead, the primary risk is the timing of Federal Reserve policy changes. While persistent inflation allows the Fed to hold rates, a rapid rise in long-term U.S. yields could threaten the trade. Another concern is crowding, as massive inflows increase vulnerability to a sudden reversal. High interest rates in regions like Latin America and Eastern Europe, supported by external factors like Middle East tensions and energy prices, continue to sustain the opportunity. Major investors remain engaged, favoring currencies like the Mexican peso, South African rand, and Turkish lira.

marsbitHace 50 min(s)

Generating Profits for Seven Consecutive Quarters, Emerging Markets Carry Trade Outperforms Everything

marsbitHace 50 min(s)

Tsinghua and Whiton Mentor-Disciple Team Cracks 40-Year-Old Mystery, Core Math All Written by GPT, Anyone Could Do It

Tsinghua and Wharton researchers have used GPT-5.6 Sol Pro to solve a 40-year-old open problem in optimization theory, proving a fundamental limit for gradient descent. They demonstrated that by adjusting only the step size sequence—without adding momentum or changing the algorithm's structure—the convergence rate of gradient descent cannot exceed Ω(T^{-1.9319}). This establishes a mathematical ceiling, showing that structural modifications are necessary to achieve the fastest possible convergence rates like O(1/T²). The core proof, involving a geometric construction based on "resisting oracles," was entirely generated through iterative interaction with the AI. The proof was then formally verified using the Lean 4 theorem prover, achieving zero "sorry" or "admit" statements, confirming its logical correctness without human intervention in the non-trivial mathematical steps. This result bridges the gap between the known O(T^{-1.2716}) rate achieved by the "silver stepsize" sequence and the newly proven impossibility bound. The work suggests a new paradigm where researchers can leverage general-purpose AI to explore and formally verify complex theoretical proofs.

marsbitHace 1 hora(s)

Tsinghua and Whiton Mentor-Disciple Team Cracks 40-Year-Old Mystery, Core Math All Written by GPT, Anyone Could Do It

marsbitHace 1 hora(s)

Trading

Spot