AI2 Releases Fully Open-Source Web Agent MolmoWeb: Controlling Web Pages Using Only "Vision"

marsbitPublished on 2026-03-26Last updated on 2026-03-26

Abstract

AI2 has released MolmoWeb, a groundbreaking, fully open-source web agent that operates solely by analyzing screenshots, marking a significant leap in vision-driven web navigation. Unlike traditional agents that rely on DOM, MolmoWeb captures and interprets visual data to make decisions—such as clicking, scrolling, or typing—making its process transparent and robust. Despite its compact size (4B and 8B parameters), MolmoWeb performs impressively: it scores 78.2% on the WebVoyager benchmark, nearing OpenAI’s proprietary o3 model (79.3%), and achieves up to 94.7% success with multiple attempts. It even surpasses Anthropic’s Claude3.7 in UI element localization. AI2 also released MolmoWebMix, a massive open dataset with 36K human-browsing tasks, over 2.2M screenshot-QA pairs, and GPT-4o-verified synthetic data. The model and data are fully available on Hugging Face and GitHub under Apache 2.0, promoting transparency and collaboration in AI development. Challenges remain in complex instructions, logins, and legal compliance.

The Allen Institute for Artificial Intelligence (AI2) recently released the groundbreaking fully open-source web agent MolmoWeb . Unlike traditional agents that rely on a webpage's underlying code (DOM), MolmoWeb makes decisions solely by reading screenshots, marking a significant leap forward in "vision-driven" web navigation technology.

Core Technology: "Seeing" Web Pages Like a Human

MolmoWeb's operating logic is very intuitive: it captures a screenshot of the current browser window, decides the next action (such as clicking, scrolling, or paging) through visual analysis, then executes it and repeats. This "what you see is what you get" model makes it more robust than traditional agents because the visual layout of a webpage is generally more stable than its underlying code, and its decision-making process is completely transparent and explainable to human users.

Performance Leap: Small Model Outperforms Giants

Despite having parameter sizes of only 4B and 8B, MolmoWeb demonstrates a "small but mighty" performance:

  • Topping the Charts: In the WebVoyager test, the 8B version scored an impressive 78.2%, not only ranking among the top open-source models but also approaching the performance of OpenAI's proprietary model o3 (79.3%).

  • Huge Potential: Research found that by running tasks multiple times and selecting the optimal result, its success rate could further jump to 94.7%.

  • Precise Localization: In UI element localization benchmark tests, it even surpassed Anthropic's Claude3.7.

Data Support: The Largest Open Dataset to Date

AI2 has not only open-sourced the model weights but also contributed a massive dataset named MolmoWebMix. This dataset contains:

  • 36,000 real browsing tasks completed by human volunteers.

  • Over 2.2 million screenshot-question-answer pairs.

  • Automated synthetic data verified by GPT-4o. Experiments show that synthetic data is even better than human trajectories at guiding the agent to find the "optimal path".

Open-Source Spirit and Future Challenges

Currently, MolmoWeb is fully available under the Apache 2.0 license on Hugging Face and GitHub. Although it still faces challenges in handling complex instructions, login authentication, and legal compliance (such as terms of service), AI2 firmly believes that only through complete transparency and community collaboration can we truly counter the data monopoly of large tech companies.

Related Questions

QWhat is the name of the fully open-source web agent released by the Allen Institute for AI (AI2) that navigates using only screenshots?

AThe web agent is called MolmoWeb.

QHow does MolmoWeb's approach to web navigation differ from traditional web agents?

AUnlike traditional agents that rely on a webpage's underlying code (DOM), MolmoWeb makes decisions by reading and analyzing screenshots, making it a 'vision-driven' technology.

QWhat was the performance score of the 8B parameter version of MolmoWeb on the WebVoyager test, and how does it compare to OpenAI's model?

AThe 8B version scored 78.2% on the WebVoyager test, which is very close to the performance of OpenAI's proprietary model o3, which scored 79.3%.

QWhat is the name of the large, open dataset released alongside MolmoWeb, and what does it contain?

AThe dataset is called MolmoWebMix. It contains 36,000 real browsing tasks completed by human volunteers, over 2.2 million screenshot-QA pairs, and automated synthetic data verified by GPT-4o.

QOn which platforms has MolmoWeb been made available, and under what license?

AMolmoWeb has been fully released on Hugging Face and GitHub under the Apache 2.0 license.

Related Reads

Attention, Bitcoin Users! Today's Hack May Be Larger Than You Think. Here's What to Do

Bitcoin users are urged to take immediate action following the disclosure of a critical security vulnerability. Hardware wallet manufacturer Coinkite has revealed a flaw in the seed phrase generation process for certain Coldcard Mk3 devices running firmware versions 4.0.1 through 5.0.3. All bitcoin addresses created from seed phrases generated on these potentially affected devices are now considered at risk. This warning coincides with an ongoing investigation into a major hack that drained approximately 1,082.65 BTC (worth roughly $70.2 million at the time) from 1,196 single-signature wallets in just 41 minutes on July 30. Security analysts believe the coordinated thefts, which used identical transaction fees and left no change, were executed by an automated tool scanning for compromised private keys. Notably, this attack occurred about 30 hours before Coinkite's public disclosure of the Coldcard vulnerability, raising concerns of a possible connection. Experts strongly recommend that impacted Coldcard Mk3 users do not consider their old seed phrase safe. The advised course of action is to generate a completely new, secure seed phrase on a trusted, modern hardware wallet and then transfer all funds to addresses derived from this new seed. Simply moving the old, potentially compromised seed to a new device is not sufficient.

cryptonews.ru4m ago

Attention, Bitcoin Users! Today's Hack May Be Larger Than You Think. Here's What to Do

cryptonews.ru4m ago

Bitcoin in August: Experts Expect Range Testing, Not a Quick Reversal

Experts anticipate Bitcoin will remain under pressure in August, lacking momentum for a sustained rally after a partial July recovery. The price, trading around $63.5k, is seen stuck in a $60-65k range. Analysts cite a challenging macro environment of high U.S. rates, persistent inflation, and a strong dollar as key headwinds, alongside record ETF outflows. Historically, August is a weak month for crypto. Experts outline three main scenarios: a base case of sideways movement between $58-68k (50% probability), a bearish drop to $50-55k (30%), or a bullish push toward $71-75k (20%). Key levels to watch are support at $60-61k and resistance at $67k. While some see current prices as attractive for long-term accumulation, most warn of potential short-term downside, with a retest of $60k or even a fall toward $53k possible. A decisive market turnaround is not expected before Q4, making August a month for testing key levels rather than a major breakout.

cryptonews.ru23m ago

Bitcoin in August: Experts Expect Range Testing, Not a Quick Reversal

cryptonews.ru23m ago

Coldcard Hardware Wallet Hacked: 594 Bitcoin Withdrawn in 25 Minutes

The Coldcard hardware wallet has been compromised, with hackers stealing approximately 594.5 Bitcoin (~$40 million) from 500 addresses in just 25 minutes. The root cause was a critical software bug, undetected for five years, which disabled the device's secure chip for generating true random numbers. This led to the creation of private keys based on predictable data like the processor's serial number, drastically reducing cryptographic security. The attackers exploited this offline by brute-forcing possible seed phrases, finding active addresses on the public ledger, and signing transactions. Initially, Coinkite (Coldcard's maker) claimed only older models were at risk but later admitted all devices running the compromised firmware were vulnerable. CEO Rodolphe Novak (NVK) apologized but ruled out financial compensation for affected users. To secure funds, owners must urgently update their firmware to specific safe versions, generate a completely new seed phrase on the updated device, and transfer all assets to new addresses created with that new seed. While a BIP-39 passphrase can help, it does not replace this migration process. Other Coinkite products like TAPSIGNER were not affected. This incident underscores that even specialized hardware requires rigorous, independent code audits, especially for cryptographic functions. It parallels past failures, like a 2006 OpenSSL bug in Debian, and raises questions about whether automated code analysis can ever fully replace human scrutiny in critical security areas.

cryptonews.ru23m ago

Coldcard Hardware Wallet Hacked: 594 Bitcoin Withdrawn in 25 Minutes

cryptonews.ru23m ago

Tether (USDT) Publishes Anticipated Financial Results for the Second Quarter! Here's Its Current Status and Assets

Tether, the world's largest stablecoin issuer, has published its Q2 2026 financial results. According to an independent audit by BDO, the company reported a net operating profit of approximately $1.5 billion and an excess of assets over liabilities totaling $4.11 billion. As of June 30, 2026, the circulating supply of USDT was roughly $184.6 billion, marking an increase of about $446 million from the prior quarter. Tether now holds over 60% of the global stablecoin market share. The $1.5 billion profit was primarily generated from U.S. Treasury bills and repurchase agreements. Total assets stood at approximately $187.75 billion against liabilities of $183.64 billion, with $183.62 billion of liabilities tied to issued digital tokens. Key actions during the quarter included a reduction of secured loans by $2.38 billion (down 15%) and an addition of 14 tons to its physical gold holdings, bringing the total to over 146 tons. CEO Paolo Ardoino reaffirmed that USDT remains fully backed by reserves despite market volatility. He also noted that the company is undergoing a comprehensive independent audit with one of the "Big Four" accounting firms.

cryptonews.ru1h ago

Tether (USDT) Publishes Anticipated Financial Results for the Second Quarter! Here's Its Current Status and Assets

cryptonews.ru1h ago

Circle Reaches Major Regulatory Milestone, Receives Trust Charter from New York Department of Financial Services (NYDFS)

Circle, a financial services and stablecoin company, announced it reached a key regulatory milestone by receiving a Limited Purpose Trust Charter from the New York State Department of Financial Services (NYDFS). This charter allows Circle to expand its operations in New York to include asset custody services, issuance of NYDFS-approved stablecoins, and management of the reserves backing them. The approval positions Circle's USDC stablecoin within a regulated framework and follows the company's earlier receipt of a national trust bank charter from the U.S. OCC. CEO Jeremy Allaire stated this achievement reflects over a decade of compliance commitment and aligns with NYDFS's high standards, comparable to those for full commercial banks, as digital dollars become central to the global financial system.

cryptonews.ru1h ago

Circle Reaches Major Regulatory Milestone, Receives Trust Charter from New York Department of Financial Services (NYDFS)

cryptonews.ru1h ago

Trading

Spot
活动图片