AI2 Releases Fully Open-Source Web Agent MolmoWeb: Controlling Web Pages Using Only "Vision"

marsbitPublished on 2026-03-26Last updated on 2026-03-26

Abstract

AI2 has released MolmoWeb, a groundbreaking, fully open-source web agent that operates solely by analyzing screenshots, marking a significant leap in vision-driven web navigation. Unlike traditional agents that rely on DOM, MolmoWeb captures and interprets visual data to make decisions—such as clicking, scrolling, or typing—making its process transparent and robust. Despite its compact size (4B and 8B parameters), MolmoWeb performs impressively: it scores 78.2% on the WebVoyager benchmark, nearing OpenAI’s proprietary o3 model (79.3%), and achieves up to 94.7% success with multiple attempts. It even surpasses Anthropic’s Claude3.7 in UI element localization. AI2 also released MolmoWebMix, a massive open dataset with 36K human-browsing tasks, over 2.2M screenshot-QA pairs, and GPT-4o-verified synthetic data. The model and data are fully available on Hugging Face and GitHub under Apache 2.0, promoting transparency and collaboration in AI development. Challenges remain in complex instructions, logins, and legal compliance.

The Allen Institute for Artificial Intelligence (AI2) recently released the groundbreaking fully open-source web agent MolmoWeb . Unlike traditional agents that rely on a webpage's underlying code (DOM), MolmoWeb makes decisions solely by reading screenshots, marking a significant leap forward in "vision-driven" web navigation technology.

Core Technology: "Seeing" Web Pages Like a Human

MolmoWeb's operating logic is very intuitive: it captures a screenshot of the current browser window, decides the next action (such as clicking, scrolling, or paging) through visual analysis, then executes it and repeats. This "what you see is what you get" model makes it more robust than traditional agents because the visual layout of a webpage is generally more stable than its underlying code, and its decision-making process is completely transparent and explainable to human users.

Performance Leap: Small Model Outperforms Giants

Despite having parameter sizes of only 4B and 8B, MolmoWeb demonstrates a "small but mighty" performance:

  • Topping the Charts: In the WebVoyager test, the 8B version scored an impressive 78.2%, not only ranking among the top open-source models but also approaching the performance of OpenAI's proprietary model o3 (79.3%).

  • Huge Potential: Research found that by running tasks multiple times and selecting the optimal result, its success rate could further jump to 94.7%.

  • Precise Localization: In UI element localization benchmark tests, it even surpassed Anthropic's Claude3.7.

Data Support: The Largest Open Dataset to Date

AI2 has not only open-sourced the model weights but also contributed a massive dataset named MolmoWebMix. This dataset contains:

  • 36,000 real browsing tasks completed by human volunteers.

  • Over 2.2 million screenshot-question-answer pairs.

  • Automated synthetic data verified by GPT-4o. Experiments show that synthetic data is even better than human trajectories at guiding the agent to find the "optimal path".

Open-Source Spirit and Future Challenges

Currently, MolmoWeb is fully available under the Apache 2.0 license on Hugging Face and GitHub. Although it still faces challenges in handling complex instructions, login authentication, and legal compliance (such as terms of service), AI2 firmly believes that only through complete transparency and community collaboration can we truly counter the data monopoly of large tech companies.

Related Questions

QWhat is the name of the fully open-source web agent released by the Allen Institute for AI (AI2) that navigates using only screenshots?

AThe web agent is called MolmoWeb.

QHow does MolmoWeb's approach to web navigation differ from traditional web agents?

AUnlike traditional agents that rely on a webpage's underlying code (DOM), MolmoWeb makes decisions by reading and analyzing screenshots, making it a 'vision-driven' technology.

QWhat was the performance score of the 8B parameter version of MolmoWeb on the WebVoyager test, and how does it compare to OpenAI's model?

AThe 8B version scored 78.2% on the WebVoyager test, which is very close to the performance of OpenAI's proprietary model o3, which scored 79.3%.

QWhat is the name of the large, open dataset released alongside MolmoWeb, and what does it contain?

AThe dataset is called MolmoWebMix. It contains 36,000 real browsing tasks completed by human volunteers, over 2.2 million screenshot-QA pairs, and automated synthetic data verified by GPT-4o.

QOn which platforms has MolmoWeb been made available, and under what license?

AMolmoWeb has been fully released on Hugging Face and GitHub under the Apache 2.0 license.

Related Reads

a16z: From Companies to DAOs, DUNA May Become the Next Generation Organizational Form

This article, "From Companies to DAOs: How DUNA Could Become the Next Organizational Form," traces the 500-year evolution of business collaboration. It begins with medieval structures like the *commenda* and Florentine *compagnia*, which exposed partners to personal risk. The modern corporation, exemplified by the Dutch East India Company (VOC), was a revolutionary leap, enabling large-scale, capital-intensive ventures by offering limited liability and reducing coordination costs. However, corporations introduced new challenges like principal-agent problems and bureaucratic overhead. The piece argues that software and internet-native protocols are now reducing these traditional overheads. Decentralized Autonomous Organizations (DAOs) emerged as a new model for coordination without centralized management. Yet, DAOs face a significant legal vacuum: they lack legal recognition, leaving members exposed to unlimited personal liability, and their tokens are vulnerable to being classified as securities under unclear regulations (e.g., the Howey Test). This has forced projects into suboptimal workarounds like offshore foundations. The article identifies the Decentralized Unincorporated Nonprofit Association (DUNA) as a potential solution. Recently legalized in states like Wyoming, the DUNA provides a legal wrapper for decentralized networks. It grants key protections—legal personality, limited liability, and perpetual existence—to a group without imposing a traditional hierarchical management structure. This allows token-holder communities to govern, hold assets, and contract as a single legal entity, aligning with their decentralized nature. While DUNA doesn't solve all governance challenges or magically resolve securities law questions, it represents a crucial step. It fills the legal recognition gap, offering a native legal form for internet-scale, decentralized collaboration and extending the separation of personal risk from organizational venture into a new domain.

marsbit3m ago

a16z: From Companies to DAOs, DUNA May Become the Next Generation Organizational Form

marsbit3m ago

2026 Mid-Year Report On-Chain RWA: Tokenized Stock Market Cap Doubles in a Year, But 90% of Rights Are Hollow Shells

The 2026 Mid-Year Report on On-Chain RWA highlights a significant growth in tokenized stock market capitalization, which nearly doubled from $951 million in March to $1.89 billion by July. However, the report reveals a fundamental contradiction in this "layer 2.5" ecosystem: products with the strongest legal foundation (like regulated U.S. infrastructure) lack liquidity and distribution, while freely tradable offshored wrapper products often lack substantive ownership rights. The increase is driven largely by a few products (SECZ, FGRS, STRCx) and platforms (Ondo, xStocks, Securitize collectively hold over 85% share). While distributed value across networks like Ethereum, Solana, and BNB Chain has grown, the market remains fragmented. Products referencing the same underlying asset (e.g., Apple stock) are distinct legal liabilities with different intermediaries and jurisdictional rules, offering varying degrees of legal claim. The report cautions that headline numbers are misleading, as they reflect changes in distributed token value—driven by issuance, conversions, and price movements—not pure investor inflows. True "canonical shares" with legal ownership, wide wallet distribution, institutional liquidity, and independent on-chain price discovery do not yet exist at scale. Tokenized treasuries show stronger product-market fit, and ETFs may be easier to scale than single stocks. The core takeaway is a trade-off: legal certainty versus liquidity and composability.

marsbit52m ago

2026 Mid-Year Report On-Chain RWA: Tokenized Stock Market Cap Doubles in a Year, But 90% of Rights Are Hollow Shells

marsbit52m ago

Attention, Bitcoin Users! Today's Hack May Be Larger Than You Think. Here's What to Do

Bitcoin users are urged to take immediate action following the disclosure of a critical security vulnerability. Hardware wallet manufacturer Coinkite has revealed a flaw in the seed phrase generation process for certain Coldcard Mk3 devices running firmware versions 4.0.1 through 5.0.3. All bitcoin addresses created from seed phrases generated on these potentially affected devices are now considered at risk. This warning coincides with an ongoing investigation into a major hack that drained approximately 1,082.65 BTC (worth roughly $70.2 million at the time) from 1,196 single-signature wallets in just 41 minutes on July 30. Security analysts believe the coordinated thefts, which used identical transaction fees and left no change, were executed by an automated tool scanning for compromised private keys. Notably, this attack occurred about 30 hours before Coinkite's public disclosure of the Coldcard vulnerability, raising concerns of a possible connection. Experts strongly recommend that impacted Coldcard Mk3 users do not consider their old seed phrase safe. The advised course of action is to generate a completely new, secure seed phrase on a trusted, modern hardware wallet and then transfer all funds to addresses derived from this new seed. Simply moving the old, potentially compromised seed to a new device is not sufficient.

cryptonews.ru2h ago

Attention, Bitcoin Users! Today's Hack May Be Larger Than You Think. Here's What to Do

cryptonews.ru2h ago

Bitcoin in August: Experts Expect Range Testing, Not a Quick Reversal

Experts anticipate Bitcoin will remain under pressure in August, lacking momentum for a sustained rally after a partial July recovery. The price, trading around $63.5k, is seen stuck in a $60-65k range. Analysts cite a challenging macro environment of high U.S. rates, persistent inflation, and a strong dollar as key headwinds, alongside record ETF outflows. Historically, August is a weak month for crypto. Experts outline three main scenarios: a base case of sideways movement between $58-68k (50% probability), a bearish drop to $50-55k (30%), or a bullish push toward $71-75k (20%). Key levels to watch are support at $60-61k and resistance at $67k. While some see current prices as attractive for long-term accumulation, most warn of potential short-term downside, with a retest of $60k or even a fall toward $53k possible. A decisive market turnaround is not expected before Q4, making August a month for testing key levels rather than a major breakout.

cryptonews.ru3h ago

Bitcoin in August: Experts Expect Range Testing, Not a Quick Reversal

cryptonews.ru3h ago

Coldcard Hardware Wallet Hacked: 594 Bitcoin Withdrawn in 25 Minutes

The Coldcard hardware wallet has been compromised, with hackers stealing approximately 594.5 Bitcoin (~$40 million) from 500 addresses in just 25 minutes. The root cause was a critical software bug, undetected for five years, which disabled the device's secure chip for generating true random numbers. This led to the creation of private keys based on predictable data like the processor's serial number, drastically reducing cryptographic security. The attackers exploited this offline by brute-forcing possible seed phrases, finding active addresses on the public ledger, and signing transactions. Initially, Coinkite (Coldcard's maker) claimed only older models were at risk but later admitted all devices running the compromised firmware were vulnerable. CEO Rodolphe Novak (NVK) apologized but ruled out financial compensation for affected users. To secure funds, owners must urgently update their firmware to specific safe versions, generate a completely new seed phrase on the updated device, and transfer all assets to new addresses created with that new seed. While a BIP-39 passphrase can help, it does not replace this migration process. Other Coinkite products like TAPSIGNER were not affected. This incident underscores that even specialized hardware requires rigorous, independent code audits, especially for cryptographic functions. It parallels past failures, like a 2006 OpenSSL bug in Debian, and raises questions about whether automated code analysis can ever fully replace human scrutiny in critical security areas.

cryptonews.ru3h ago

Coldcard Hardware Wallet Hacked: 594 Bitcoin Withdrawn in 25 Minutes

cryptonews.ru3h ago

Trading

Spot
活动图片