Month-old Ethereum client bug blamed for Prysm outage

cointelegraphPublished on 2025-12-15Last updated on 2025-12-15

Abstract

A month-old bug in the Prysm Ethereum client caused a significant outage on December 4, leading to resource exhaustion and performance degradation. The issue occurred when Prysm nodes processed attestations from out-of-sync nodes, forcing them to recompute expensive state transitions. Although the bug had been present on testnets for a month prior, it wasn’t triggered until the mainnet incident. The network experienced an 18.5% missed slot rate for over 42 epochs, with participation dropping to 75%, resulting in approximately 382 ETH lost in attestation rewards. A patch was quickly deployed to resolve the issue. Developers emphasized that client diversity helped prevent a more severe impact, as Prysm only holds a 17.6% share of the network. However, they warned that Lighthouse, the dominant client with 52.6% share, is nearing a dangerous threshold where a single bug could finalize an invalid chain.

Prysm has revealed that a bug introduced in a testnet a month before Ethereum’s Fusaka upgrade was the cause of an Ethereum node validation issue that affected its client earlier this month.

Ethereum developer Terence Tsao posted a post-mortem on Sunday detailing the Fusaka mainnet Prysm incident that impacted the network on Dec. 4.

Prysm nodes experienced “resource exhaustion” when processing attestations from out-of-sync nodes, it stated. This caused Prysm to replay past epoch blocks and recompute expensive state transitions, resulting in a significant impact on performance due to the excessive workload.

The post-mortem revealed that the bug had been present on testnets for a month before the incident, but wasn’t triggered.

“The bug was introduced in Prysm PR 15965 and deployed to testnets a month before the incident without the trigger happening.”

Testnets are designed to identify bugs, but they aren’t a foolproof method.

In May 2023 — a month after the Shanghai hard fork — Ethereum developers were sent into a frenzy when the network temporarily lost transaction finality for around 25 minutes, then again for over an hour the following day, before the blockchain recovered on its own.

Prysm has been patched

Instead of using the current head state, Prysm regenerated prior states from scratch, creating a massive computational burden.

For more than 42 epochs, the network saw an 18.5% missed slot rate with participation dropping to 75% while validators lost approximately 382 Ether (ETH) in attestation rewards, it said.

Related: Vitalik Buterin says Ethereum can handle temporary loss of finality

Node operators were instructed to deploy a temporary solution while developers worked on an update patch for Prysm clients.

Client diversity saved the day

The incident could have been much worse if it had hit Ethereum’s dominant consensus client, Lighthouse, said developers.

Offchain Labs’ Prysm is the second-largest Ethereum client with a 17.6% share, according to ClientDiversity.

“Client diversity prevented a noticeable impact on Ethereum users. A client with more than 1/3rd of the network would have caused a temporary loss in finality and more missed blocks.”

However, the incident highlighted that Lighthouse is dangerously close to the two-thirds threshold where a single client bug could finalize an invalid chain.

Lighthouse currently has a client share of 52.6%, down from around 56% at the time of the incident.

Ethereum developers are pushing for more client diversity. Source: ClientDiversity


Magazine: Big questions: Would Bitcoin survive a 10-year power outage?

Related Reads

King of Death Spiral Financing: Riding the Bitcoin Wave to the Top of Japan's Capital Market

Michael Lerch, a controversial American investor and head of Evolution Financial Group, has become the dominant force in Japan's "death spiral" financing market. His fund, Evo, is the largest buyer of floating-strike equity warrants, a financing tool for distressed small-cap companies that provides quick cash but causes significant shareholder dilution. In 2025, Evo's transactions, which accounted for over 80% of the market, exceeded ¥1 trillion ($63 billion), driven largely by a massive deal with Metaplanet. The hotel operator used the funds to purchase over $2 billion in Bitcoin, causing its stock to skyrocket before crashing 80%. While some CEOs credit Lerch with saving their companies, others accuse his aggressive tactics of damaging shareholder value and corporate reputations. Based in Nevada, Lerch built his empire through arbitrage and a vast network, but his methods have drawn regulatory scrutiny and legal challenges, including a London court ruling against his firm in a dispute with a former employee. The rise of Evo highlights the dilemma for Japan's many small listed companies that rely on such high-cost financing to survive.

marsbit3m ago

King of Death Spiral Financing: Riding the Bitcoin Wave to the Top of Japan's Capital Market

marsbit3m ago

Google Is Really Anxious, Launches Deep Research Agent Late at Night with MCP Support and Native Charts

Google has launched two new AI research agents, Deep Research and Deep Research Max, built on the Gemini 3.1 Pro model. These agents are designed for enterprise and developer use, accessible via API, and support integration of open web data with private enterprise information through a single API call. They also feature native chart and infographic generation within reports and support the Model Context Protocol (MCP) to connect with third-party data sources securely. Deep Research is optimized for speed and lower latency, while Deep Research Max prioritizes depth and extended reasoning, making it suitable for asynchronous workflows like overnight analysis tasks. The agents are not available to general Gemini app users, including Pro subscribers, and are currently offered as a paid preview through the Interactions API. The update aims to position Google competitively against rivals like OpenAI and Anthropic in the AI research and analysis tool space, particularly targeting high-value sectors such as finance and consulting. Enhanced capabilities include multi-modal input support, collaborative planning features, and improved performance on benchmarks, though comparative data with competitors should be interpreted cautiously due to potential methodological differences.

marsbit4m ago

Google Is Really Anxious, Launches Deep Research Agent Late at Night with MCP Support and Native Charts

marsbit4m ago

Booking.com Founder Jeff Hoffman: How Web3 and AI Are Reshaping the Trillion-Dollar Social Travel Market

Booking.com co-founder Jeff Hoffman discusses how Web3 and AI are set to reshape the multitrillion-dollar social travel market. Hoffman, who co-founded Priceline and helped build Booking.com into a global giant, emphasizes that the current travel industry remains fragmented and inefficient. He argues that Web3 introduces structural disruption by enabling direct connections, transparent economics, and faster settlements, while AI powers personalization and smarter recommendations. Hoffman highlights key trends driving this shift: travelers demand flexible, authentic rewards instead of expiring points; digital and borderless payments are becoming standard; and people trust communities more than ads. He joined Staynex not for its Web3 label but because it aligns with these trends—integrating booking, rewards, AI, and payments into a seamless ecosystem. Looking ahead, Hoffman predicts travel will evolve from transactional to relational, with blockchain enabling transparent rewards and cross-border payments, and AI delivering hyper-personalized experiences. He believes the most valuable platforms will own the network around payments, loyalty, and community, making social travel one of the most underestimated opportunities in Web3.

marsbit38m ago

Booking.com Founder Jeff Hoffman: How Web3 and AI Are Reshaping the Trillion-Dollar Social Travel Market

marsbit38m ago

Is It Illegal for Crypto KOLs to Run Paid Communities and Sell Courses?

Summary: This article, authored by lawyer Shao Shiwei, addresses the legal risks faced by crypto Key Opinion Leaders (KOLs) who offer paid communities and courses related to cryptocurrency trading. The author explains that while there is no direct licensing system for such activities in China, authorities still reference virtual currency policies in enforcement. KOL activities are categorized into three risk levels: 1. Low-risk: Historical trend analysis and basic education without real-time guidance. 2. Medium-risk: Providing daily market analysis and ambiguous suggestions (e.g., "this position is worth watching"). 3. High-risk: Explicit trading signals, specific buy/sell points, or profit-sharing arrangements. Key legal risks include civil liability (e.g., user lawsuits over losses) and criminal charges (e.g., fraud accusations if users suffer losses). Even disclaimers like "not investment advice" may not protect KOLs, as courts focus on the实质 (substance) of the content. The article emphasizes that risk depends on whether content is perceived as operational guidance, its influence on user decisions, and payment structures. KOLs are advised to assess their specific business models carefully.

marsbit1h ago

Is It Illegal for Crypto KOLs to Run Paid Communities and Sell Courses?

marsbit1h ago

Cook's Curtain Call and Ternus Takes the Helm: The Disruption and Reboot of Apple's 4 Trillion Dollar Empire

Tim Cook has officially announced he will step down as CEO of Apple in September, transitioning to executive chairman after a 15-year tenure during which he grew the company’s market value from around $350 billion to nearly $4 trillion. He will be succeeded by John Ternus, a 50-year-old hardware engineering veteran who has been groomed for the role through increasing public visibility and internal responsibility. Ternus’s appointment signals a strategic shift toward hardware and engineering leadership, with Johny Srouji—head of Apple Silicon—taking on an expanded role as Chief Hardware Officer. This consolidation aims to strengthen Apple’s core technological capabilities. However, Cook’s departure highlights a significant unresolved issue: Apple’s delayed and fragmented approach to artificial intelligence. Despite early efforts, such as hiring John Giannandrea from Google in 2018, Apple’s AI initiatives—particularly around Siri—have struggled with internal restructuring and reliance on external partnerships, including with Google. The transition comes at a critical moment as Apple faces paradigm shifts with the rise of artificial general intelligence (ASI). The company’s closed ecosystem of hardware, software, and services—once a major advantage—now presents challenges in adapting to an AI-centric world where intelligence may matter more than the device itself. Ternus must quickly articulate a clear AI strategy, possibly starting at WWDC, to reassure markets and redefine Apple’s role in a new technological era. His task is not only to maintain Apple’s operational excellence but also to reinvigorate its capacity to innovate and lead in the age of AI.

marsbit2h ago

Cook's Curtain Call and Ternus Takes the Helm: The Disruption and Reboot of Apple's 4 Trillion Dollar Empire

marsbit2h ago

Trading

Spot
Futures

Hot Articles

Discussions

Welcome to the HTX Community. Here, you can stay informed about the latest platform developments and gain access to professional market insights. Users' opinions on the price of ETH (ETH) are presented below.

活动图片

Top Questions

Hot Categoriesright-arrow

Hot Tagsright-arrow