AI2 Releases Fully Open-Source Web Agent MolmoWeb: Controlling Web Pages Using Only "Vision"

marsbitОпубликовано 2026-03-26Обновлено 2026-03-26

Введение

AI2 has released MolmoWeb, a groundbreaking, fully open-source web agent that operates solely by analyzing screenshots, marking a significant leap in vision-driven web navigation. Unlike traditional agents that rely on DOM, MolmoWeb captures and interprets visual data to make decisions—such as clicking, scrolling, or typing—making its process transparent and robust. Despite its compact size (4B and 8B parameters), MolmoWeb performs impressively: it scores 78.2% on the WebVoyager benchmark, nearing OpenAI’s proprietary o3 model (79.3%), and achieves up to 94.7% success with multiple attempts. It even surpasses Anthropic’s Claude3.7 in UI element localization. AI2 also released MolmoWebMix, a massive open dataset with 36K human-browsing tasks, over 2.2M screenshot-QA pairs, and GPT-4o-verified synthetic data. The model and data are fully available on Hugging Face and GitHub under Apache 2.0, promoting transparency and collaboration in AI development. Challenges remain in complex instructions, logins, and legal compliance.

The Allen Institute for Artificial Intelligence (AI2) recently released the groundbreaking fully open-source web agent MolmoWeb . Unlike traditional agents that rely on a webpage's underlying code (DOM), MolmoWeb makes decisions solely by reading screenshots, marking a significant leap forward in "vision-driven" web navigation technology.

Core Technology: "Seeing" Web Pages Like a Human

MolmoWeb's operating logic is very intuitive: it captures a screenshot of the current browser window, decides the next action (such as clicking, scrolling, or paging) through visual analysis, then executes it and repeats. This "what you see is what you get" model makes it more robust than traditional agents because the visual layout of a webpage is generally more stable than its underlying code, and its decision-making process is completely transparent and explainable to human users.

Performance Leap: Small Model Outperforms Giants

Despite having parameter sizes of only 4B and 8B, MolmoWeb demonstrates a "small but mighty" performance:

  • Topping the Charts: In the WebVoyager test, the 8B version scored an impressive 78.2%, not only ranking among the top open-source models but also approaching the performance of OpenAI's proprietary model o3 (79.3%).

  • Huge Potential: Research found that by running tasks multiple times and selecting the optimal result, its success rate could further jump to 94.7%.

  • Precise Localization: In UI element localization benchmark tests, it even surpassed Anthropic's Claude3.7.

Data Support: The Largest Open Dataset to Date

AI2 has not only open-sourced the model weights but also contributed a massive dataset named MolmoWebMix. This dataset contains:

  • 36,000 real browsing tasks completed by human volunteers.

  • Over 2.2 million screenshot-question-answer pairs.

  • Automated synthetic data verified by GPT-4o. Experiments show that synthetic data is even better than human trajectories at guiding the agent to find the "optimal path".

Open-Source Spirit and Future Challenges

Currently, MolmoWeb is fully available under the Apache 2.0 license on Hugging Face and GitHub. Although it still faces challenges in handling complex instructions, login authentication, and legal compliance (such as terms of service), AI2 firmly believes that only through complete transparency and community collaboration can we truly counter the data monopoly of large tech companies.

Связанные с этим вопросы

QWhat is the name of the fully open-source web agent released by the Allen Institute for AI (AI2) that navigates using only screenshots?

AThe web agent is called MolmoWeb.

QHow does MolmoWeb's approach to web navigation differ from traditional web agents?

AUnlike traditional agents that rely on a webpage's underlying code (DOM), MolmoWeb makes decisions by reading and analyzing screenshots, making it a 'vision-driven' technology.

QWhat was the performance score of the 8B parameter version of MolmoWeb on the WebVoyager test, and how does it compare to OpenAI's model?

AThe 8B version scored 78.2% on the WebVoyager test, which is very close to the performance of OpenAI's proprietary model o3, which scored 79.3%.

QWhat is the name of the large, open dataset released alongside MolmoWeb, and what does it contain?

AThe dataset is called MolmoWebMix. It contains 36,000 real browsing tasks completed by human volunteers, over 2.2 million screenshot-QA pairs, and automated synthetic data verified by GPT-4o.

QOn which platforms has MolmoWeb been made available, and under what license?

AMolmoWeb has been fully released on Hugging Face and GitHub under the Apache 2.0 license.

Похожее

Coinbase Launches Crypto-Backed USDC Loans For UK Users In Latest Expansion

Coinbase has expanded its crypto-backed USDC lending service to UK residents, allowing them to use Bitcoin (BTC), Ethereum (ETH), and Coinbase Wrapped Staked Ether (cbETH) as collateral. Powered by the on-chain protocol Morpho on the Base network, the service enables users to borrow up to $5 million in USDC without selling their crypto holdings. Collateral is locked in a smart contract until the loan is repaid, with liquidation triggered if the loan-to-value ratio exceeds a threshold. This follows the service’s successful US launch, where originations surpassed $2.17 billion. The move is part of Coinbase’s broader expansion in the UK, including savings accounts and DEX trading, and aligns with its efforts to integrate crypto into traditional finance, such as recently offering crypto-backed mortgages.

bitcoinist1 ч. назад

Coinbase Launches Crypto-Backed USDC Loans For UK Users In Latest Expansion

bitcoinist1 ч. назад

Capital Flow Analysis Shows Ozak AI Absorbing Liquidity From BTC, ETH, and SOL During Market Pullbacks

Capital flow analysis indicates that during recent market pullbacks, liquidity is rotating from major cryptocurrencies like BTC, ETH, and SOL into Ozak AI, an early-stage AI project. Priced at $0.014, Ozak AI has raised over $6.8 million and sold more than 1.17 billion tokens in its presale, showing steady and sustained capital absorption rather than hype-driven spikes. Analysts describe this shift as strategic redeployment, not panic selling, driven by Ozak AI’s relative valuation efficiency, AI-native utility (including Prediction Agents and EigenLayer integration), and favorable risk-reward timing. The trend reflects a structural rotation into high-growth AI infrastructure projects during large-cap consolidation, with Ozak AI emerging as a preferred alternative for asymmetric returns.

TheNewsCrypto2 ч. назад

Capital Flow Analysis Shows Ozak AI Absorbing Liquidity From BTC, ETH, and SOL During Market Pullbacks

TheNewsCrypto2 ч. назад

DeFi Falls into the Most Dangerous Prisoner's Dilemma in History

ChainCatcher author Gu Yu analyzes the severe fallout from the Kelp DAO exploit, which has triggered a crisis of confidence across the DeFi ecosystem, implicating major projects like Aave, LayerZero, and Arbitrum. The attack, attributed to the Lazarus Group, exploited a single-validator (1/1) configuration in LayerZero’s decentralized verification network (DVN), leading to over $40M in losses. LayerZero’s official report claimed its protocol “operated as expected” and deflected blame onto Kelp DAO’s configuration choices, sparking backlash from developers and researchers who criticized its avoidance of responsibility. Meanwhile, Arbitrum’s security committee intervened by using a privileged transaction to freeze and recover ~30,766 ETH ($71M) from the hacker’s address—a move praised for pragmatism but criticized for undermining Layer2 decentralization ideals. The incident has intensified debates on cross-chain bridge security, Layer2 centralization risks, and the tension between practical security measures and decentralized principles. With Aave facing ~$200M in bad debt and no clear compensation plan from Kelp DAO or LayerZero, the situation reflects a dangerous “prisoner’s dilemma” where key players prioritize self-interest over collective trust, threatening foundational DeFi narratives.

marsbit2 ч. назад

DeFi Falls into the Most Dangerous Prisoner's Dilemma in History

marsbit2 ч. назад

Not Speculation but a Necessity: The 4 Unique Values of Prediction Markets

Polymarket's recent $4 billion funding round and soaring valuation of $15 billion highlight the explosive growth of prediction markets, with trading volume reaching $25.7 billion in March 2026—a 10.6% monthly increase. This analysis argues that prediction markets serve critical non-speculative functions, positioning them as essential tools rather than mere gambling platforms. Prediction markets offer four unique values: entertainment consumption, insurance-like protection, risk hedging, and truth discovery. Firstly, they stimulate economic activity by engaging users in event-based betting, similar to the broader sports industry. Secondly, they act as a form of decentralized insurance, allowing users to hedge against specific, well-defined risks (e.g., weather events) transparently and without traditional overhead costs. Thirdly, institutions and individuals use these markets to hedge against geopolitical and commodity price risks, as demonstrated during the U.S.-Iran conflict and the launch of 24/7 commodity markets on platforms like Kalshi. Finally, prediction markets counter media bias by aggregating crowd-sourced information, often achieving 30% higher accuracy than surveys due to users' vested interests. Experts like Bitwise’s Jeff Park and SIG’s Jeff Yass emphasize the markets' role in risk transfer and financial innovation. As these platforms evolve, they are poised to become trillion-dollar markets, offering more reliable, decentralized mechanisms for information pricing and risk management.

marsbit2 ч. назад

Not Speculation but a Necessity: The 4 Unique Values of Prediction Markets

marsbit2 ч. назад

Not Speculation but a Necessity: The 4 Unique Values of Prediction Markets

Polymarket's recent $4 billion funding round, valuing it at $15 billion, highlights the explosive growth of prediction markets, with trading volume surging from $8.7 billion to $25.7 billion in just a few months. This article argues that prediction markets are not merely speculative gambling platforms but serve four core functions. First, they act as a form of entertainment and consumption, stimulating economic activity similar to the sports industry. Second, they provide a form of limited insurance, allowing users to hedge against specific, well-defined risks (e.g., weather events) in a transparent, cost-effective manner without traditional insurance overhead. Third, they are powerful tools for risk hedging, enabling both individuals and institutions to mitigate exposure to volatile events like commodity price swings and geopolitical crises. Finally, prediction markets can function as "truth machines," aggregating crowd wisdom to counter media narratives and biases, with some claims suggesting a 30% higher accuracy over traditional surveys. The piece concludes that the role and potential value of prediction markets are far greater than commonly perceived, positioning them as a significant future market.

Odaily星球日报2 ч. назад

Not Speculation but a Necessity: The 4 Unique Values of Prediction Markets

Odaily星球日报2 ч. назад

Торговля

Спот
Фьючерсы
活动图片

Популярные категорииright-arrow

Популярные тегиright-arrow