Anthropic Cries Wolf: Is the AGI Threat Real, or Just an IPO Story?

marsbitPublished on 2026-06-05Last updated on 2026-06-05

Abstract

Anthropic has published an article titled "When AI builds itself," discussing the emerging concept of "recursive self-improvement," where AI begins to actively participate in designing, training, testing, and optimizing its own subsequent versions. The company presents internal data showing that by May 2026, over 80% of code merged into its codebase was written by Claude, its AI model. Claude's capabilities have expanded to handling complex, open-ended engineering tasks, achieving a 76% success rate in such areas, and even contributing to research processes, such as optimizing code performance and conducting AI safety experiments. Anthropic outlines an evolution from human-driven development to AI-assisted workflows, culminating in the current stage where AI agents can autonomously write, run, and delegate code. The company cautions that the path toward a "closed loop," where AI continuously improves itself, is becoming visible. It calls for coordinated global mechanisms to potentially slow or pause frontier AI development to allow safety research and societal structures to catch up. However, the timing of this warning coincides with Anthropic's preparations for an IPO, framing the narrative not just as a safety concern but also as a demonstration of Claude's advanced capabilities and its integral role in accelerating Anthropic's own R&D—creating a potential "flywheel" effect for competitive advantage. This contrasts with OpenAI's recent, more policy-oriented discussion of ...

By | Alphabet AI

Anthropic published a lengthy article last night titled "When AI builds itself," which sounds like a science fiction novel by Asimov, and indeed deals with a sci-fi concept: recursive self-improvement.

Simply put, in the past, human researchers wrote code, ran experiments, and trained models to make AI stronger. But if AI starts to participate in designing, training, testing, and optimizing its own successors, then the speed of AI progress is no longer driven solely by humans—it may begin to "self-evolve."

To this end, Anthropic made a plea:

"We believe it would be beneficial for the world if there were an option to slow down or temporarily pause frontier AI development, allowing societal structures and alignment research to catch up with technological progress."

This statement sounds like a safety warning, but in the context of Anthropic preparing for an IPO, it's hard not to see it as another kind of narrative setup: Claude is so good, it's even starting to create the next generation of Claude itself.

A New Storm Has Emerged

To illustrate that AI is increasingly involved in AI research and development itself, Anthropic presented substantial internal data.

For instance, as of May 2026, over 80% of the code merged into Anthropic's codebase was written by Claude. Before the release of Claude Code, this number was only in the single digits.

By the second quarter of 2026, according to Anthropic's statistics, the daily volume of code merged by engineers was about 8 times higher than in 2024.

More notable than the code volume is that Claude is handling more open-ended engineering problems.

Anthropic stated in the article that over the past year, the frequency with which employees had to correct Claude, steer it back on track, or take over tasks mid-way has been steadily declining. This change is happening not only for simple tasks but also for the most complex, open-ended tasks.

So-called open-ended tasks are problems without clear instructions. For example, a system crash, a training task failure—issues where even engineers themselves don't know what the solution looks like initially and have to troubleshoot and make judgments on the fly.

These types of tasks historically relied most heavily on human experience. Yet, in those most open-ended tasks, Claude's success rate reached 76% by May 2026, a 50 percentage point increase within six months.

Not just writing code, Anthropic also uses Claude for code review—checking for bugs, security vulnerabilities, and other defects. Their retrospective analysis found that if every code change in the past had undergone automated review by Claude, approximately one-third of the bugs that caused incidents on claude.ai could have been caught before deployment.

Going a step further, Claude has begun to participate in the research process.

Anthropic has a standard test: give Claude code for training a small model and ask it to make the code run faster without altering the results. In May 2025, Claude Opus 4 could achieve about a 3x speedup; by April 2026, Claude Mythos Preview had pushed that number to approximately 52x.

Anthropic also mentioned an open-ended AI safety research case. They posed a question to a Claude-powered agent: Can a weaker model reliably supervise a stronger model?

This process involved proposing hypotheses, testing them, sharing findings with parallel agents, and iterating repeatedly.

Two human researchers spent a week bridging about 23% of the gap; Claude, with roughly 800 cumulative hours and about $18,000 in compute costs, bridged 97%.

This result certainly has limitations—the problem was chosen by humans, the scoring criteria were human-defined, and the findings haven't been fully migrated to production-scale models. But it still illustrates that Claude can now, within a research framework defined by humans, design experiments, execute them, and iterate on its own.

Furthermore, when human researchers "go down the wrong path," Claude can suggest a better next step.

Anthropic took 129 internal Claude Code research sessions where human researchers and Claude worked together on open-ended research problems. Anthropic identified points where "the human later proved to have taken a detour," gave the context up to that point to different versions of Claude, and asked it what it would suggest doing next. Then, another Claude judge, aware of the full session outcome, judged which was better: the model's suggestion or the human's choice at the time.

The results showed that at those points where the human researcher was later shown to have had room for improvement, Claude became increasingly able to propose a better next step.

In the past, AI model progress was primarily driven by human researchers and engineers. Humans decided what experiments to run, wrote the code, trained the models, and pushed forward AI's capabilities.

Now, more and more links in this chain are being taken over by Claude.

Anthropic presented a very intuitive stage diagram:

From 2021 to 2023, Anthropic was no different from a typical tech company—humans writing code and documentation on laptops.

From 2023 to 2025, chatbots began entering workflows. Engineers had models generate code snippets, then copied them into editors.

From 2025 to 2026, programming agents emerged. Claude began autonomously writing and modifying code, sometimes even completing entire files independently.

Today, agents can run code on their own and delegate hours-long work to other agents.

Looking ahead is the stage Anthropic is genuinely concerned about: the closed loop.

If this day arrives, subsequent versions of Claude might be continuously improved by Claude itself—this is recursive self-improvement.

Anthropic phrased it cautiously: we haven't reached that point yet, and recursive self-improvement isn't inevitable. But it still emphasizes that the path leading to that step is beginning to become visible.

That's why Anthropic discusses slowing down, even pausing, at the end of the article. Its meaning isn't that all AI companies should shut down immediately, but rather that if the risks of AI self-improvement continue to rise in the future, frontier labs need a coordinated, verifiable deceleration mechanism.

In other words, the "singularity" is approaching, and humanity must impose controls.

Unstoppable Claude

On the surface, this is a very forward-looking safety document. Anthropic is talking about recursive self-improvement, about AI potentially improving itself faster and faster, and about the need for human society to prepare deceleration and pause mechanisms in advance.

But placed in the context of Anthropic preparing for an IPO, this article takes on another layer of meaning.

In a way, Anthropic's recent moves resemble that annoyingly smug top student in class—it genuinely has the skills, but it's also quite pretentious.

What it wants to say isn't just "we have a very strong Claude"; a step beyond that, it wants to say "Claude is helping us build an even stronger Claude."

If Anthropic were merely selling a model or a tool, it would struggle to completely escape horizontal comparisons: Anthropic has Claude, OpenAI has GPT; Anthropic has Claude Code, OpenAI has Codex; Anthropic competes for enterprise clients, OpenAI competes for enterprise clients. The competition between the two companies is very tight, seeing who can tell the bigger story to the market.

It's worth noting that just three days ago, OpenAI wrote in a document about frontier AI governance:

"We are already seeing early signs of recursive self-improvement in today's systems: AI development itself is being accelerated by AI.

This will intensify competitive pressures among developers and nations, and create governance challenges that existing institutions are not equipped to handle."

Three days later, Anthropic says: The path for Claude towards recursive self-improvement is beginning to become visible.

If Claude develops as it hopes, this wouldn't be an ordinary product narrative—it would become a research and development flywheel.

Claude writes code, runs experiments, optimizes training processes, which in turn reduces incidents in Anthropic's own products… Once this system is up and running, Claude isn't just a product from Anthropic; it's a crucial production tool for Anthropic itself.

Users see the Claude product; enterprise customers buy Claude's capabilities. But what Anthropic truly wants the capital markets to notice is: Claude is already embedded in the underlying processes of frontier model development; it's been placed inside Anthropic's engine room.

Capital markets love flywheel stories, promising endless prosperity: A stronger Claude allows Anthropic's engineers to merge more code; more code enables faster product and infrastructure iteration; faster iteration allows researchers to run more experiments; more experiments in turn help the next generation of Claude become stronger. Once the next generation Claude is stronger, it continues to accelerate Anthropic's R&D.

Claude's iteration pace also supports this flywheel. Looking at public release timelines, from 2023 to early 2025, Claude's major model updates were mostly on a three-to-four month cycle. But with Claude 4, Anthropic's model updates have noticeably intensified.

Claude 4 was released in May 2025, Opus 4.1 in August, Sonnet 4.5 in September, Haiku 4.5 in October, Opus 4.5 in November.

In 2026, Opus 4.6 was released on February 5, Sonnet 4.6 on February 17, Opus 4.7 on April 15, and Opus 4.8 on May 28. The gap between Opus 4.7 and Opus 4.8 was only 42 days.

Anthropic, on the surface, is saying "this could be very dangerous, we need to prepare the brakes in advance," but it's simultaneously implying: "We've seen what happens when the accelerator is pressed."

The subtlety of the IPO narrative lies here. It describes the risks as significant while also elevating its own technological position.

Not every AI company is qualified to discuss recursive self-improvement. You first need to make the outside world believe your AI is already part of the AI R&D process to have the standing to say this might require global coordination.

OpenAI: How Could This Happen?

As mentioned earlier, just before Anthropic published this lengthy article, OpenAI had already put recursive self-improvement on the table.

But the two companies' narratives are quite different.

OpenAI's document, "Democratic Governance of Frontier AI," is a policy blueprint for Washington. It's concerned not with "how models get stronger," but with how to constrain frontier AI if it continues to surge ahead.

Most of the content in that report isn't suitable for detailed discussion here, but one key line stands out: OpenAI said that in today's systems, early signs of recursive self-improvement are already visible.

This line and Anthropic's lengthy article point in the same direction.

It's just that OpenAI talks about institutions, while Anthropic talks about itself.

OpenAI's point is: AI development is too fast; existing governance structures may not keep up, so a new set of rules is needed.

Anthropic directly showcases that system, telling the market: Claude is already in our R&D process, so we see the path to AI self-acceleration.

This move is quite clever. One imagines the grumbling inside OpenAI—this is practically idea theft! We were here first!

Just joking, but OpenAI really needs to step up its game and quickly bring GPT 5.6 to the table.

Related Questions

QWhat is the core concern of Anthropic's article 'When AI builds itself'?

AThe article's core concern is the potential for recursive self-improvement in AI, where AI begins to design, train, test, and optimize its own successor versions, potentially leading to an AI evolution that is no longer solely driven by human effort.

QWhat significant performance statistic does Anthropic cite to show Claude's growing role in AI development?

AAnthropic states that as of May 2026, over 80% of the code merged into its codebase is written by Claude.

QHow does the author interpret the timing of Anthropic's publication on AI risks?

AThe author suggests that the article, coming at the time when Anthropic is preparing for an IPO, can be seen as a narrative prelude to highlight their technological advantage, framing it as both a safety warning and a display of a powerful 'virtuous cycle' where Claude helps build a stronger Claude.

QWhat is the key difference between OpenAI's and Anthropic's approach to discussing 'recursive self-improvement'?

AOpenAI's recent document framed it as a broad governance challenge requiring new institutional rules. Anthropic's article, however, presented it through the specific lens of its own development process with Claude, showcasing internal data to illustrate the visible path towards such a capability.

QWhat future stage in AI development is Anthropic particularly worried about, as described in the article?

AAnthropic is particularly concerned about the potential arrival of a 'closed loop' stage, where a future version of Claude could continuously improve itself, leading to true recursive self-improvement.

Related Reads

Near Returns to the AI Stage: Transformation into a Public Chain Due to 'Payroll Difficulties,' Agent and Privacy Emerge as New Growth Narratives

NEAR Returns to AI Origins: From Payroll Struggles to Blockchain, Now Focusing on AI Agents and Privacy NEAR Protocol's journey began not with grand blockchain ambitions, but from a practical hurdle: its AI startup founders, including Transformer paper co-author Illia Polosukhin, couldn't efficiently pay international developers in 2017. This led them to pivot and build a high-performance, scalable blockchain. After years navigating various crypto narratives like sharding and cross-chain interoperability, NEAR is now leveraging its AI roots to re-enter the AI arena. A key driver is its "NEAR Intents" layer, which abstracts complex cross-chain transactions. Users simply state their goal (e.g., swap BTC for ETH), and a solver network finds the optimal route. This system has processed over $20B in cross-chain volume, generating significant fee revenue. A major growth area is private transactions via "Confidential Intents/Swaps," which hide trade details until settlement to protect against MEV and front-running. Remarkably, private swaps recently accounted for over 40% of NEAR's transaction volume, highlighting strong demand but also potential regulatory scrutiny. With its AI-founder pedigree, NEAR is positioning itself at the intersection of blockchain, AI agents, and privacy, aiming to become infrastructure for the emerging agent economy while navigating the challenges of its rapid adoption.

marsbit1h ago

Near Returns to the AI Stage: Transformation into a Public Chain Due to 'Payroll Difficulties,' Agent and Privacy Emerge as New Growth Narratives

marsbit1h ago

From Ethereum to AI's 'CROPS': What Exactly is This Set of 'Slow Variables' That Vitalik Repeatedly Emphasizes?

In recent discussions, Vitalik Buterin has frequently emphasized the concept of "CROPS," a framework defining core values for Ethereum's development. CROPS stands for Censorship Resistance, Capture Resistance, Open Source, Privacy, and Security. Initially outlined in the Ethereum Foundation's "EF Mandate," it represents a commitment to user sovereignty, ensuring that the network resists external control, remains open, protects privacy, and prioritizes security. The relevance of CROPS extends beyond Ethereum's foundational principles, becoming crucial in the context of AI integration. As AI agents begin handling wallet operations and automated transactions, the risk increases that users may cede control over their digital assets, privacy, and intentions to centralized AI service providers. A "CROPS AI" would therefore emphasize local execution where possible, privacy-preserving remote model calls (e.g., using zero-knowledge proofs), and transparent, verifiable processes to maintain user agency. Vitalik highlights a significant convergence between "CROPS Ethereum access layer" and "CROPS AI." Both address the same fundamental challenge: how users can access powerful services—be it blockchain data via RPCs or AI models—without exposing sensitive information or relinquishing ultimate control. This intersection points toward a future digital entry point that is more private, secure, and user-controlled. Ultimately, CROPS is not merely an abstract ideal but a practical guidepost. It steers development—from protocol resilience and wallet design to AI agent safety—towards a future where users retain self-sovereignty even as digital systems grow more complex and powerful. In an era of accelerating AI adoption, these "slow variables" of censorship resistance, openness, privacy, and security may define Ethereum's enduring value.

marsbit1h ago

From Ethereum to AI's 'CROPS': What Exactly is This Set of 'Slow Variables' That Vitalik Repeatedly Emphasizes?

marsbit1h ago

Silicon Valley 'Startup Guru' Steve Hoffman: Web3 + AI Could Be a Trap

Silicon Valley investor and "Godfather of Startups" Steve Hoffman warns that combining Web3 with AI is likely a trap, not a promising venture. In an interview, Hoffman argues that while AI is a foundational technology touching all industries, Web3 adds complexity, friction, and regulatory risk without solving mainstream consumer or business needs. He advises founders to focus on deep, specialized applications where startups can out-iterate giants, rather than on generic features easily replicated by large tech companies. Hoffman observes that Silicon Valley will lead foundational AI research, while China excels at rapid, large-scale application and commercialization, particularly in robotics. He stresses that AI-driven autonomous agents capable of collaborative, multi-step tasks are 2-4 years away, which will cause significant job displacement. The solution is not to slow AI but to redesign business models around human-AI collaboration and reform social systems like education and retraining. For startups, Hoffman recommends focusing on vertical, expertise-heavy domains to build defensibility. He sees major opportunities in AI fraud detection and cybersecurity. Key founder mindsets include systemic thinking over feature-focus, relentless customer centricity, building adaptive teams, and deeply understanding AI's capabilities and limits. Hoffman is also leading a non-profit initiative to establish university centers aimed at training future leaders in responsible, human-value-aligned AI innovation.

marsbit3h ago

Silicon Valley 'Startup Guru' Steve Hoffman: Web3 + AI Could Be a Trap

marsbit3h ago

Token Inefficient, Economy Tokenless

The article "Tokens Aren't Economical, Economics Aren't Tokenized" analyzes a pivotal shift in the AI industry from a technology-driven narrative to one dominated by capital efficiency. It highlights two concurrent trends: a severe capital shortage due to the exorbitant and recurring costs of compute (e.g., OpenAI's high burn rate) and a wave of corporate spin-offs where major tech companies are separating their AI units (like Kuaishou's Kling and Baidu's Kunlunxin). The core argument is that AI's "anti-internet" business model, where user growth increases costs rather than profits, has created a disconnect between high valuations and actual cash flow. Spin-offs address this by allowing AI assets to be valued independently. Within a parent company, they are seen as cost centers, but as standalone entities, they are priced based on their growth potential and scarcity in the primary market, leading to massive valuation premiums (e.g., Kling's estimated value tripling post-spin-off). The industry is at an inflection point, moving from "model worship" to "value realization." The competition is evolving from a pure compute (GPU) race to a broader focus on systemic efficiency and full-stack engineering (involving CPUs and orchestration) to achieve viable commercialization. The year 2026 is framed as a critical moment where the industry must definitively answer how to economically translate AI capability into tangible business value, reshaping the sector's future power structure.

marsbit3h ago

Token Inefficient, Economy Tokenless

marsbit3h ago

Trading

Spot
Futures

Hot Articles

How to Buy NIGHT

Welcome to HTX.com! We've made purchasing Midnight (NIGHT) simple and convenient. Follow our step-by-step guide to embark on your crypto journey.Step 1: Create Your HTX AccountUse your email or phone number to sign up for a free account on HTX. Experience a hassle-free registration journey and unlock all features.Get My AccountStep 2: Go to Buy Crypto and Choose Your Payment MethodCredit/Debit Card: Use your Visa or Mastercard to buy Midnight (NIGHT) instantly.Balance: Use funds from your HTX account balance to trade seamlessly.Third Parties: We've added popular payment methods such as Google Pay and Apple Pay to enhance convenience.P2P: Trade directly with other users on HTX.Over-the-Counter (OTC): We offer tailor-made services and competitive exchange rates for traders.Step 3: Store Your Midnight (NIGHT)After purchasing your Midnight (NIGHT), store it in your HTX account. Alternatively, you can send it elsewhere via blockchain transfer or use it to trade other cryptocurrencies.Step 4: Trade Midnight (NIGHT)Easily trade Midnight (NIGHT) on HTX's spot market. Simply access your account, select your trading pair, execute your trades, and monitor in real-time. We offer a user-friendly experience for both beginners and seasoned traders.

3.4k Total ViewsPublished 2025.12.08Updated 2026.06.02

How to Buy NIGHT

Discussions

Welcome to the HTX Community. Here, you can stay informed about the latest platform developments and gain access to professional market insights. Users' opinions on the price of NIGHT (NIGHT) are presented below.

活动图片