AI2 Releases Fully Open-Source Web Agent MolmoWeb: Controlling Web Pages Using Only "Vision"

marsbitPublished on 2026-03-26Last updated on 2026-03-26

Abstract

AI2 has released MolmoWeb, a groundbreaking, fully open-source web agent that operates solely by analyzing screenshots, marking a significant leap in vision-driven web navigation. Unlike traditional agents that rely on DOM, MolmoWeb captures and interprets visual data to make decisions—such as clicking, scrolling, or typing—making its process transparent and robust. Despite its compact size (4B and 8B parameters), MolmoWeb performs impressively: it scores 78.2% on the WebVoyager benchmark, nearing OpenAI’s proprietary o3 model (79.3%), and achieves up to 94.7% success with multiple attempts. It even surpasses Anthropic’s Claude3.7 in UI element localization. AI2 also released MolmoWebMix, a massive open dataset with 36K human-browsing tasks, over 2.2M screenshot-QA pairs, and GPT-4o-verified synthetic data. The model and data are fully available on Hugging Face and GitHub under Apache 2.0, promoting transparency and collaboration in AI development. Challenges remain in complex instructions, logins, and legal compliance.

The Allen Institute for Artificial Intelligence (AI2) recently released the groundbreaking fully open-source web agent MolmoWeb . Unlike traditional agents that rely on a webpage's underlying code (DOM), MolmoWeb makes decisions solely by reading screenshots, marking a significant leap forward in "vision-driven" web navigation technology.

Core Technology: "Seeing" Web Pages Like a Human

MolmoWeb's operating logic is very intuitive: it captures a screenshot of the current browser window, decides the next action (such as clicking, scrolling, or paging) through visual analysis, then executes it and repeats. This "what you see is what you get" model makes it more robust than traditional agents because the visual layout of a webpage is generally more stable than its underlying code, and its decision-making process is completely transparent and explainable to human users.

Performance Leap: Small Model Outperforms Giants

Despite having parameter sizes of only 4B and 8B, MolmoWeb demonstrates a "small but mighty" performance:

  • Topping the Charts: In the WebVoyager test, the 8B version scored an impressive 78.2%, not only ranking among the top open-source models but also approaching the performance of OpenAI's proprietary model o3 (79.3%).

  • Huge Potential: Research found that by running tasks multiple times and selecting the optimal result, its success rate could further jump to 94.7%.

  • Precise Localization: In UI element localization benchmark tests, it even surpassed Anthropic's Claude3.7.

Data Support: The Largest Open Dataset to Date

AI2 has not only open-sourced the model weights but also contributed a massive dataset named MolmoWebMix. This dataset contains:

  • 36,000 real browsing tasks completed by human volunteers.

  • Over 2.2 million screenshot-question-answer pairs.

  • Automated synthetic data verified by GPT-4o. Experiments show that synthetic data is even better than human trajectories at guiding the agent to find the "optimal path".

Open-Source Spirit and Future Challenges

Currently, MolmoWeb is fully available under the Apache 2.0 license on Hugging Face and GitHub. Although it still faces challenges in handling complex instructions, login authentication, and legal compliance (such as terms of service), AI2 firmly believes that only through complete transparency and community collaboration can we truly counter the data monopoly of large tech companies.

Related Questions

QWhat is the name of the fully open-source web agent released by the Allen Institute for AI (AI2) that navigates using only screenshots?

AThe web agent is called MolmoWeb.

QHow does MolmoWeb's approach to web navigation differ from traditional web agents?

AUnlike traditional agents that rely on a webpage's underlying code (DOM), MolmoWeb makes decisions by reading and analyzing screenshots, making it a 'vision-driven' technology.

QWhat was the performance score of the 8B parameter version of MolmoWeb on the WebVoyager test, and how does it compare to OpenAI's model?

AThe 8B version scored 78.2% on the WebVoyager test, which is very close to the performance of OpenAI's proprietary model o3, which scored 79.3%.

QWhat is the name of the large, open dataset released alongside MolmoWeb, and what does it contain?

AThe dataset is called MolmoWebMix. It contains 36,000 real browsing tasks completed by human volunteers, over 2.2 million screenshot-QA pairs, and automated synthetic data verified by GPT-4o.

QOn which platforms has MolmoWeb been made available, and under what license?

AMolmoWeb has been fully released on Hugging Face and GitHub under the Apache 2.0 license.

Related Reads

TON/GRAM price prediction – Telegram’s Gram wallet rollout lifts token: Can the rally continue?

Telegram founder Pavel Durov announced the rollout of a native, non-custodial Gram wallet across the messaging app, describing it as potentially the largest deployment of a non-custodial wallet in history. The move aims to bring instant, zero-fee crypto transactions to Telegram's over one billion users, integrating digital assets directly into the platform. The news triggered a more than 7% price increase for TON/GRAM, pushing the token to around $1.53. However, it remains below its May peak near $2.80, facing immediate resistance around $1.60. Analysts suggest a break above this level could improve sentiment toward $1.80, while a failure to hold gains might see a retest of support near $1.40.

ambcrypto17m ago

TON/GRAM price prediction – Telegram’s Gram wallet rollout lifts token: Can the rally continue?

ambcrypto17m ago

How 40.8M staked ETH could strengthen Ethereum’s edge over Bitcoin

Ethereum is showing signs of strengthening against Bitcoin, driven by two key trends. First, significant whale accumulation is occurring, with large holders not just buying ETH but immediately staking 100% of it. This locks away supply, signaling long-term conviction. Ethereum's total staked ETH has reached a record 40.8 million, with a growing entry queue and 33.5% of supply now staked, further tightening liquid availability. Second, Ethereum's DeFi ecosystem is experiencing robust activity, with WETH whale transactions at a multi-year high and Total Value Locked (TVL) rising sharply. These converging factors—reduced supply from staking and increased on-chain demand—are translating into technical strength. The ETH/BTC ratio has broken above resistance and is approaching the 0.03 level, setting the stage for potential continued outperformance against Bitcoin.

ambcrypto50m ago

How 40.8M staked ETH could strengthen Ethereum’s edge over Bitcoin

ambcrypto50m ago

Global Stock Market's Storm Center: South Korea's Stock Market De-leveraging Is Largely Complete

Storm's Eye: South Korean Market De-leveraging Nears Completion The recent sharp correction in South Korean equities, with the KOSPI index dropping 32% from its June high, has been a key trigger for global tech stock volatility. The core driver was not a fundamental shift but a forced de-leveraging process within the market's unique structure, which is now largely complete. Two main leverage channels amplified the sell-off: 1. **Leveraged ETFs:** Their size, proportionally four times larger than in the U.S., peaked near $50 billion. Their mandatory daily rebalancing mechanism created a vicious cycle of "price drop → forced selling → further drop." Approximately 75% of this excess has been unwound, shrinking to $26 billion, with regulatory curbs now blocking new inflows. 2. **Hedge Fund Leverage:** Using swaps to magnify exposure, hedge funds saw their net long positioning fall by over 50% from peak levels. The most intense phase of this institutional de-leveraging is over. In contrast, **retail margin debt** poses minimal systemic risk. At 0.5% of market cap, it is far lower than in the U.S. or China, lacks automatic triggers, and is concentrated in smaller stocks. The conclusion: the high-leverage structures most prone to "chain-reaction selling" have been substantially cleared. The market is transitioning from a liquidity-driven crash to one priced more on fundamentals. The article argues that the AI trend—centered on Korean memory chips—remains intact. This episode represents a painful but necessary clearing of crowded trades, not the end of the AI revolution. For investors, the key question is conviction in the long-term AI direction; if the trend is real, current volatility is a cost of entry, not a terminal risk.

链捕手1h ago

Global Stock Market's Storm Center: South Korea's Stock Market De-leveraging Is Largely Complete

链捕手1h ago

92.9% of crypto tokens launched since 2024 trade below TGE price: CryptoRank

A recent analysis by CryptoRank reveals that 92.9% of cryptocurrency tokens launched since 2024 and with a market capitalization exceeding $100 million are now trading below their initial Token Generation Event (TGE) price. The study, examining 113 projects, found only eight—or 7.1%—remain profitable, with the median return across the dataset being a stark -95.7%. Top performers include Hyperliquid (HYPE), which gained 1,519%, followed by Ondo Finance (ONDO), EverValue Coin (EVA), and Midnight Network (NIGHT). The findings indicate a market that has become highly selective, with capital concentrating on a small number of projects demonstrating real utility, adoption, or ecosystem growth. This contrasts with previous cycles where new tokens often saw sustained rallies. Investors are now scrutinizing factors like tokenomics, circulating supply, and unlock schedules more closely. This trend may influence future token launches, prompting projects to prioritize sustainable distribution models and long-term growth over aggressive initial valuations, as high fully diluted valuations (FDVs) and large future token unlocks often lead to significant price pressure.

ambcrypto1h ago

92.9% of crypto tokens launched since 2024 trade below TGE price: CryptoRank

ambcrypto1h ago

The Eternal Fragments of Money: Third-Party Payment Has No First Principle

"The Enduring Fragments of Money: Third-Party Payments Lack a First Principle" Stripe is reportedly attempting to acquire PayPal, marking a significant shift reminiscent of PayPal's merger with the original X.com 30 years ago. The article analyzes Stripe's strategic challenges and the broader payments industry landscape. Despite its initial success with a developer-friendly API model, Stripe missed its optimal IPO window during the pandemic and has since seen its valuation decline. Its attempts to expand through acquisitions and new ventures, particularly in stablecoins (like its OUSD project) and Agent-focused payments (ACP/MPP protocols), have faced headwinds. The author argues that the payment industry remains highly fragmented and is ultimately an adjunct to the traditional banking system. This structure limits the potential for any single player, including Stripe, to achieve complete dominance. While stablecoins and the future rise of autonomous Agent economies present potential growth avenues, they are not yet mainstream and still require integration with the existing financial system. For now, Agent-based transactions are largely used for speculative "volume boosting" rather than substantive business applications. Stripe's current move to acquire PayPal is seen as an attempt to bolster its weak consumer-facing (C-side) business after its stablecoin-focused strategies faltered. Meanwhile, PayPal is described as structurally outdated, unable to revive itself through new products like Venmo or PYUSD. The future of payments may lie not in payments themselves but in value-added services like more efficient settlement networks. The author suggests that companies like Stripe and Circle, which are building their own blockchains (Tempo, Arc) and stablecoins, are positioning themselves to eventually profit from high-efficiency settlement systems. These new networks could potentially bypass some traditional banking layers. In conclusion, the article posits that third-party payment is a perpetually fragmented battlefield where scale alone cannot ensure victory. Players must find new models, focusing on efficiency to compete with the entrenched banking system. Stripe's acquisition of PayPal represents a bet on this uncertain future.

链捕手1h ago

The Eternal Fragments of Money: Third-Party Payment Has No First Principle

链捕手1h ago

Trading

Spot
活动图片