Anthropic Cries Wolf: Is the AGI Threat Real, or Just an IPO Story?

marsbitPublished on 2026-06-05Last updated on 2026-06-05

Abstract

Anthropic has published an article titled "When AI builds itself," discussing the emerging concept of "recursive self-improvement," where AI begins to actively participate in designing, training, testing, and optimizing its own subsequent versions. The company presents internal data showing that by May 2026, over 80% of code merged into its codebase was written by Claude, its AI model. Claude's capabilities have expanded to handling complex, open-ended engineering tasks, achieving a 76% success rate in such areas, and even contributing to research processes, such as optimizing code performance and conducting AI safety experiments. Anthropic outlines an evolution from human-driven development to AI-assisted workflows, culminating in the current stage where AI agents can autonomously write, run, and delegate code. The company cautions that the path toward a "closed loop," where AI continuously improves itself, is becoming visible. It calls for coordinated global mechanisms to potentially slow or pause frontier AI development to allow safety research and societal structures to catch up. However, the timing of this warning coincides with Anthropic's preparations for an IPO, framing the narrative not just as a safety concern but also as a demonstration of Claude's advanced capabilities and its integral role in accelerating Anthropic's own R&D—creating a potential "flywheel" effect for competitive advantage. This contrasts with OpenAI's recent, more policy-oriented discussion of ...

By | Alphabet AI

Anthropic published a lengthy article last night titled "When AI builds itself," which sounds like a science fiction novel by Asimov, and indeed deals with a sci-fi concept: recursive self-improvement.

Simply put, in the past, human researchers wrote code, ran experiments, and trained models to make AI stronger. But if AI starts to participate in designing, training, testing, and optimizing its own successors, then the speed of AI progress is no longer driven solely by humans—it may begin to "self-evolve."

To this end, Anthropic made a plea:

"We believe it would be beneficial for the world if there were an option to slow down or temporarily pause frontier AI development, allowing societal structures and alignment research to catch up with technological progress."

This statement sounds like a safety warning, but in the context of Anthropic preparing for an IPO, it's hard not to see it as another kind of narrative setup: Claude is so good, it's even starting to create the next generation of Claude itself.

A New Storm Has Emerged

To illustrate that AI is increasingly involved in AI research and development itself, Anthropic presented substantial internal data.

For instance, as of May 2026, over 80% of the code merged into Anthropic's codebase was written by Claude. Before the release of Claude Code, this number was only in the single digits.

By the second quarter of 2026, according to Anthropic's statistics, the daily volume of code merged by engineers was about 8 times higher than in 2024.

More notable than the code volume is that Claude is handling more open-ended engineering problems.

Anthropic stated in the article that over the past year, the frequency with which employees had to correct Claude, steer it back on track, or take over tasks mid-way has been steadily declining. This change is happening not only for simple tasks but also for the most complex, open-ended tasks.

So-called open-ended tasks are problems without clear instructions. For example, a system crash, a training task failure—issues where even engineers themselves don't know what the solution looks like initially and have to troubleshoot and make judgments on the fly.

These types of tasks historically relied most heavily on human experience. Yet, in those most open-ended tasks, Claude's success rate reached 76% by May 2026, a 50 percentage point increase within six months.

Not just writing code, Anthropic also uses Claude for code review—checking for bugs, security vulnerabilities, and other defects. Their retrospective analysis found that if every code change in the past had undergone automated review by Claude, approximately one-third of the bugs that caused incidents on claude.ai could have been caught before deployment.

Going a step further, Claude has begun to participate in the research process.

Anthropic has a standard test: give Claude code for training a small model and ask it to make the code run faster without altering the results. In May 2025, Claude Opus 4 could achieve about a 3x speedup; by April 2026, Claude Mythos Preview had pushed that number to approximately 52x.

Anthropic also mentioned an open-ended AI safety research case. They posed a question to a Claude-powered agent: Can a weaker model reliably supervise a stronger model?

This process involved proposing hypotheses, testing them, sharing findings with parallel agents, and iterating repeatedly.

Two human researchers spent a week bridging about 23% of the gap; Claude, with roughly 800 cumulative hours and about $18,000 in compute costs, bridged 97%.

This result certainly has limitations—the problem was chosen by humans, the scoring criteria were human-defined, and the findings haven't been fully migrated to production-scale models. But it still illustrates that Claude can now, within a research framework defined by humans, design experiments, execute them, and iterate on its own.

Furthermore, when human researchers "go down the wrong path," Claude can suggest a better next step.

Anthropic took 129 internal Claude Code research sessions where human researchers and Claude worked together on open-ended research problems. Anthropic identified points where "the human later proved to have taken a detour," gave the context up to that point to different versions of Claude, and asked it what it would suggest doing next. Then, another Claude judge, aware of the full session outcome, judged which was better: the model's suggestion or the human's choice at the time.

The results showed that at those points where the human researcher was later shown to have had room for improvement, Claude became increasingly able to propose a better next step.

In the past, AI model progress was primarily driven by human researchers and engineers. Humans decided what experiments to run, wrote the code, trained the models, and pushed forward AI's capabilities.

Now, more and more links in this chain are being taken over by Claude.

Anthropic presented a very intuitive stage diagram:

From 2021 to 2023, Anthropic was no different from a typical tech company—humans writing code and documentation on laptops.

From 2023 to 2025, chatbots began entering workflows. Engineers had models generate code snippets, then copied them into editors.

From 2025 to 2026, programming agents emerged. Claude began autonomously writing and modifying code, sometimes even completing entire files independently.

Today, agents can run code on their own and delegate hours-long work to other agents.

Looking ahead is the stage Anthropic is genuinely concerned about: the closed loop.

If this day arrives, subsequent versions of Claude might be continuously improved by Claude itself—this is recursive self-improvement.

Anthropic phrased it cautiously: we haven't reached that point yet, and recursive self-improvement isn't inevitable. But it still emphasizes that the path leading to that step is beginning to become visible.

That's why Anthropic discusses slowing down, even pausing, at the end of the article. Its meaning isn't that all AI companies should shut down immediately, but rather that if the risks of AI self-improvement continue to rise in the future, frontier labs need a coordinated, verifiable deceleration mechanism.

In other words, the "singularity" is approaching, and humanity must impose controls.

Unstoppable Claude

On the surface, this is a very forward-looking safety document. Anthropic is talking about recursive self-improvement, about AI potentially improving itself faster and faster, and about the need for human society to prepare deceleration and pause mechanisms in advance.

But placed in the context of Anthropic preparing for an IPO, this article takes on another layer of meaning.

In a way, Anthropic's recent moves resemble that annoyingly smug top student in class—it genuinely has the skills, but it's also quite pretentious.

What it wants to say isn't just "we have a very strong Claude"; a step beyond that, it wants to say "Claude is helping us build an even stronger Claude."

If Anthropic were merely selling a model or a tool, it would struggle to completely escape horizontal comparisons: Anthropic has Claude, OpenAI has GPT; Anthropic has Claude Code, OpenAI has Codex; Anthropic competes for enterprise clients, OpenAI competes for enterprise clients. The competition between the two companies is very tight, seeing who can tell the bigger story to the market.

It's worth noting that just three days ago, OpenAI wrote in a document about frontier AI governance:

"We are already seeing early signs of recursive self-improvement in today's systems: AI development itself is being accelerated by AI.

This will intensify competitive pressures among developers and nations, and create governance challenges that existing institutions are not equipped to handle."

Three days later, Anthropic says: The path for Claude towards recursive self-improvement is beginning to become visible.

If Claude develops as it hopes, this wouldn't be an ordinary product narrative—it would become a research and development flywheel.

Claude writes code, runs experiments, optimizes training processes, which in turn reduces incidents in Anthropic's own products… Once this system is up and running, Claude isn't just a product from Anthropic; it's a crucial production tool for Anthropic itself.

Users see the Claude product; enterprise customers buy Claude's capabilities. But what Anthropic truly wants the capital markets to notice is: Claude is already embedded in the underlying processes of frontier model development; it's been placed inside Anthropic's engine room.

Capital markets love flywheel stories, promising endless prosperity: A stronger Claude allows Anthropic's engineers to merge more code; more code enables faster product and infrastructure iteration; faster iteration allows researchers to run more experiments; more experiments in turn help the next generation of Claude become stronger. Once the next generation Claude is stronger, it continues to accelerate Anthropic's R&D.

Claude's iteration pace also supports this flywheel. Looking at public release timelines, from 2023 to early 2025, Claude's major model updates were mostly on a three-to-four month cycle. But with Claude 4, Anthropic's model updates have noticeably intensified.

Claude 4 was released in May 2025, Opus 4.1 in August, Sonnet 4.5 in September, Haiku 4.5 in October, Opus 4.5 in November.

In 2026, Opus 4.6 was released on February 5, Sonnet 4.6 on February 17, Opus 4.7 on April 15, and Opus 4.8 on May 28. The gap between Opus 4.7 and Opus 4.8 was only 42 days.

Anthropic, on the surface, is saying "this could be very dangerous, we need to prepare the brakes in advance," but it's simultaneously implying: "We've seen what happens when the accelerator is pressed."

The subtlety of the IPO narrative lies here. It describes the risks as significant while also elevating its own technological position.

Not every AI company is qualified to discuss recursive self-improvement. You first need to make the outside world believe your AI is already part of the AI R&D process to have the standing to say this might require global coordination.

OpenAI: How Could This Happen?

As mentioned earlier, just before Anthropic published this lengthy article, OpenAI had already put recursive self-improvement on the table.

But the two companies' narratives are quite different.

OpenAI's document, "Democratic Governance of Frontier AI," is a policy blueprint for Washington. It's concerned not with "how models get stronger," but with how to constrain frontier AI if it continues to surge ahead.

Most of the content in that report isn't suitable for detailed discussion here, but one key line stands out: OpenAI said that in today's systems, early signs of recursive self-improvement are already visible.

This line and Anthropic's lengthy article point in the same direction.

It's just that OpenAI talks about institutions, while Anthropic talks about itself.

OpenAI's point is: AI development is too fast; existing governance structures may not keep up, so a new set of rules is needed.

Anthropic directly showcases that system, telling the market: Claude is already in our R&D process, so we see the path to AI self-acceleration.

This move is quite clever. One imagines the grumbling inside OpenAI—this is practically idea theft! We were here first!

Just joking, but OpenAI really needs to step up its game and quickly bring GPT 5.6 to the table.

Related Questions

QWhat is the core concern of Anthropic's article 'When AI builds itself'?

AThe article's core concern is the potential for recursive self-improvement in AI, where AI begins to design, train, test, and optimize its own successor versions, potentially leading to an AI evolution that is no longer solely driven by human effort.

QWhat significant performance statistic does Anthropic cite to show Claude's growing role in AI development?

AAnthropic states that as of May 2026, over 80% of the code merged into its codebase is written by Claude.

QHow does the author interpret the timing of Anthropic's publication on AI risks?

AThe author suggests that the article, coming at the time when Anthropic is preparing for an IPO, can be seen as a narrative prelude to highlight their technological advantage, framing it as both a safety warning and a display of a powerful 'virtuous cycle' where Claude helps build a stronger Claude.

QWhat is the key difference between OpenAI's and Anthropic's approach to discussing 'recursive self-improvement'?

AOpenAI's recent document framed it as a broad governance challenge requiring new institutional rules. Anthropic's article, however, presented it through the specific lens of its own development process with Claude, showcasing internal data to illustrate the visible path towards such a capability.

QWhat future stage in AI development is Anthropic particularly worried about, as described in the article?

AAnthropic is particularly concerned about the potential arrival of a 'closed loop' stage, where a future version of Claude could continuously improve itself, leading to true recursive self-improvement.

Related Reads

Silicon Valley 'Startup Guru' Steve Hoffman: Web3 + AI Could Be a Trap

Silicon Valley investor and "Godfather of Startups" Steve Hoffman warns that combining Web3 with AI is likely a trap, not a promising venture. In an interview, Hoffman argues that while AI is a foundational technology touching all industries, Web3 adds complexity, friction, and regulatory risk without solving mainstream consumer or business needs. He advises founders to focus on deep, specialized applications where startups can out-iterate giants, rather than on generic features easily replicated by large tech companies. Hoffman observes that Silicon Valley will lead foundational AI research, while China excels at rapid, large-scale application and commercialization, particularly in robotics. He stresses that AI-driven autonomous agents capable of collaborative, multi-step tasks are 2-4 years away, which will cause significant job displacement. The solution is not to slow AI but to redesign business models around human-AI collaboration and reform social systems like education and retraining. For startups, Hoffman recommends focusing on vertical, expertise-heavy domains to build defensibility. He sees major opportunities in AI fraud detection and cybersecurity. Key founder mindsets include systemic thinking over feature-focus, relentless customer centricity, building adaptive teams, and deeply understanding AI's capabilities and limits. Hoffman is also leading a non-profit initiative to establish university centers aimed at training future leaders in responsible, human-value-aligned AI innovation.

marsbit1h ago

Silicon Valley 'Startup Guru' Steve Hoffman: Web3 + AI Could Be a Trap

marsbit1h ago

Token Inefficient, Economy Tokenless

The article "Tokens Aren't Economical, Economics Aren't Tokenized" analyzes a pivotal shift in the AI industry from a technology-driven narrative to one dominated by capital efficiency. It highlights two concurrent trends: a severe capital shortage due to the exorbitant and recurring costs of compute (e.g., OpenAI's high burn rate) and a wave of corporate spin-offs where major tech companies are separating their AI units (like Kuaishou's Kling and Baidu's Kunlunxin). The core argument is that AI's "anti-internet" business model, where user growth increases costs rather than profits, has created a disconnect between high valuations and actual cash flow. Spin-offs address this by allowing AI assets to be valued independently. Within a parent company, they are seen as cost centers, but as standalone entities, they are priced based on their growth potential and scarcity in the primary market, leading to massive valuation premiums (e.g., Kling's estimated value tripling post-spin-off). The industry is at an inflection point, moving from "model worship" to "value realization." The competition is evolving from a pure compute (GPU) race to a broader focus on systemic efficiency and full-stack engineering (involving CPUs and orchestration) to achieve viable commercialization. The year 2026 is framed as a critical moment where the industry must definitively answer how to economically translate AI capability into tangible business value, reshaping the sector's future power structure.

marsbit1h ago

Token Inefficient, Economy Tokenless

marsbit1h ago

Crossing the 'Memory Wall': The Wafer-Level Revolution and Computing Power Routes in the AI Inference Era

In 2026, a historic shift occurred in AI as major cloud providers' inference spending surpassed training spending for the first time, signaling a move from "building large models" to "using large models." This shifts the core challenge from computing power to the "memory wall"—the bottleneck of data movement (model weights, activations, KV Cache) between external DRAM and processors, where energy and latency from data transfer far exceed computation itself. Companies like Nvidia face GPU idle time due to bandwidth limits. In contrast, Cerebras Systems adopts a radical "wafer-scale" approach with its Wafer-Scale Engine (WSE). Instead of cutting a silicon wafer into many chips, Cerebras uses almost the entire wafer as one massive chip (WSE-3). This design provides 44GB of on-chip SRAM, delivering memory bandwidth thousands of times higher than traditional HBM (e.g., 21 PB/s vs. Nvidia B200). For LLM inference, weights are streamed layer-by-layer from external MemoryX storage to the chip, avoiding HBM bottlenecks. This results in token generation speeds 1.5–5 times faster than Nvidia's B200 in some models and significant advantages in first-token latency and long-context tasks. Additionally, Cerebras's architecture offers much lower interconnect power consumption (0.15 pJ/bit vs. GPU's ~10 pJ/bit). However, Cerebras faces challenges: SRAM scaling has slowed with advanced nodes, limiting future capacity gains; the chip requires specialized liquid cooling and custom software stacks; and its external I/O bandwidth (150 GB/s) is low compared to NVLink, hindering multi-system scaling for very large models. Competition is intensifying. Major players are pursuing three paths: 1) Developing proprietary inference ASICs (e.g., Google TPU, Microsoft Maia), 2) Leveraging advanced packaging (e.g., TSMC's SoW) to democratize wafer-scale-like integration, potentially eroding Cerebras's process advantage within a few years, and 3) Exploring optical interconnects for ultimate bandwidth. Commercially, Cerebras is transitioning from a hardware vendor to a service provider, facing the immense challenge of building high-power, specialized data centers to meet large contracts (e.g., 250MW/year from 2026–2028). In conclusion, the AI inference era presents a fundamental architectural trade-off. Cerebras opts for extreme physical optimization for low-latency, single-task performance, while Nvidia prioritizes versatility and massive cluster throughput. The path forward remains uncertain, with technology and business models still evolving in the race toward advanced AI.

marsbit1h ago

Crossing the 'Memory Wall': The Wafer-Level Revolution and Computing Power Routes in the AI Inference Era

marsbit1h ago

Has Bitcoin's 'Rebound Ended', Officially Entering the Late Bear Market Phase?

**Title: Has Bitcoin's Rebound Ended, Entering the Late Bear Market Phase?** **Summary:** Bitcoin's price has declined by 13% this week, signaling a potential return to late-stage bear market conditions. The price fell to around $67k, positioned between the Realized Price and Realized Cap Weighted Average. For the first time since early 2022, the Short-Term Holder cost basis has dropped below this key average, confirming a hallmark of late-cycle bear markets. Profitability metrics have collapsed sharply. The 7-day average of the Realized Profit/Loss ratio plummeted from a local high of 3.16 to 0.29, mirroring the February panic sell-off. Critically, the 90-day average never breached the threshold of 2, indicating the recent rally to $82k was a bear market bounce, not a structural shift. Realized losses surged to $1.35 billion daily, with $770 million coming from Long-Term Holders selling at a loss. This accelerating redistribution of supply from weak to strong hands is a necessary but ongoing process for a market bottom. The rally stalled almost precisely at the aggregate cost basis (~$83k) of US spot Bitcoin ETF investors, turning that level into strong resistance and leaving the average ETF holder underwater again. Spot market flows have turned decisively negative, showing sellers are dominating order books despite the price drop. While a significant futures long liquidation event cleared over $400 million in leverage, providing a potential reset, sustained spot demand is yet to materialize. Options markets continue to price in higher future volatility (Implied Volatility) than recent price action (Realized Volatility) has shown, with a persistent skew towards put options, indicating ongoing demand for downside protection. In conclusion, multiple metrics point to a fragile market structure. Resistance at the ETF cost basis, accelerating realized losses, dominant spot selling, and cautious options pricing all suggest the bear market trend persists. A sustainable recovery likely requires a resurgence of spot demand, ETF holders returning to profit, and a clear reduction in selling pressure.

marsbit1h ago

Has Bitcoin's 'Rebound Ended', Officially Entering the Late Bear Market Phase?

marsbit1h ago

Trading

Spot
Futures

Hot Articles

How to Buy NIGHT

Welcome to HTX.com! We've made purchasing Midnight (NIGHT) simple and convenient. Follow our step-by-step guide to embark on your crypto journey.Step 1: Create Your HTX AccountUse your email or phone number to sign up for a free account on HTX. Experience a hassle-free registration journey and unlock all features.Get My AccountStep 2: Go to Buy Crypto and Choose Your Payment MethodCredit/Debit Card: Use your Visa or Mastercard to buy Midnight (NIGHT) instantly.Balance: Use funds from your HTX account balance to trade seamlessly.Third Parties: We've added popular payment methods such as Google Pay and Apple Pay to enhance convenience.P2P: Trade directly with other users on HTX.Over-the-Counter (OTC): We offer tailor-made services and competitive exchange rates for traders.Step 3: Store Your Midnight (NIGHT)After purchasing your Midnight (NIGHT), store it in your HTX account. Alternatively, you can send it elsewhere via blockchain transfer or use it to trade other cryptocurrencies.Step 4: Trade Midnight (NIGHT)Easily trade Midnight (NIGHT) on HTX's spot market. Simply access your account, select your trading pair, execute your trades, and monitor in real-time. We offer a user-friendly experience for both beginners and seasoned traders.

3.4k Total ViewsPublished 2025.12.08Updated 2026.06.02

How to Buy NIGHT

Discussions

Welcome to the HTX Community. Here, you can stay informed about the latest platform developments and gain access to professional market insights. Users' opinions on the price of NIGHT (NIGHT) are presented below.

活动图片