AI2 Releases Fully Open-Source Web Agent MolmoWeb: Controlling Web Pages Using Only "Vision"

marsbit2026-03-26 tarihinde yayınlandı2026-03-26 tarihinde güncellendi

Özet

AI2 has released MolmoWeb, a groundbreaking, fully open-source web agent that operates solely by analyzing screenshots, marking a significant leap in vision-driven web navigation. Unlike traditional agents that rely on DOM, MolmoWeb captures and interprets visual data to make decisions—such as clicking, scrolling, or typing—making its process transparent and robust. Despite its compact size (4B and 8B parameters), MolmoWeb performs impressively: it scores 78.2% on the WebVoyager benchmark, nearing OpenAI’s proprietary o3 model (79.3%), and achieves up to 94.7% success with multiple attempts. It even surpasses Anthropic’s Claude3.7 in UI element localization. AI2 also released MolmoWebMix, a massive open dataset with 36K human-browsing tasks, over 2.2M screenshot-QA pairs, and GPT-4o-verified synthetic data. The model and data are fully available on Hugging Face and GitHub under Apache 2.0, promoting transparency and collaboration in AI development. Challenges remain in complex instructions, logins, and legal compliance.

The Allen Institute for Artificial Intelligence (AI2) recently released the groundbreaking fully open-source web agent MolmoWeb . Unlike traditional agents that rely on a webpage's underlying code (DOM), MolmoWeb makes decisions solely by reading screenshots, marking a significant leap forward in "vision-driven" web navigation technology.

Core Technology: "Seeing" Web Pages Like a Human

MolmoWeb's operating logic is very intuitive: it captures a screenshot of the current browser window, decides the next action (such as clicking, scrolling, or paging) through visual analysis, then executes it and repeats. This "what you see is what you get" model makes it more robust than traditional agents because the visual layout of a webpage is generally more stable than its underlying code, and its decision-making process is completely transparent and explainable to human users.

Performance Leap: Small Model Outperforms Giants

Despite having parameter sizes of only 4B and 8B, MolmoWeb demonstrates a "small but mighty" performance:

Topping the Charts: In the WebVoyager test, the 8B version scored an impressive 78.2%, not only ranking among the top open-source models but also approaching the performance of OpenAI's proprietary model o3 (79.3%).
Huge Potential: Research found that by running tasks multiple times and selecting the optimal result, its success rate could further jump to 94.7%.
Precise Localization: In UI element localization benchmark tests, it even surpassed Anthropic's Claude3.7.

Data Support: The Largest Open Dataset to Date

AI2 has not only open-sourced the model weights but also contributed a massive dataset named MolmoWebMix. This dataset contains:

36,000 real browsing tasks completed by human volunteers.
Over 2.2 million screenshot-question-answer pairs.
Automated synthetic data verified by GPT-4o. Experiments show that synthetic data is even better than human trajectories at guiding the agent to find the "optimal path".

Open-Source Spirit and Future Challenges

Currently, MolmoWeb is fully available under the Apache 2.0 license on Hugging Face and GitHub. Although it still faces challenges in handling complex instructions, login authentication, and legal compliance (such as terms of service), AI2 firmly believes that only through complete transparency and community collaboration can we truly counter the data monopoly of large tech companies.

İlgili Sorular

QWhat is the name of the fully open-source web agent released by the Allen Institute for AI (AI2) that navigates using only screenshots?

AThe web agent is called MolmoWeb.

QHow does MolmoWeb's approach to web navigation differ from traditional web agents?

AUnlike traditional agents that rely on a webpage's underlying code (DOM), MolmoWeb makes decisions by reading and analyzing screenshots, making it a 'vision-driven' technology.

QWhat was the performance score of the 8B parameter version of MolmoWeb on the WebVoyager test, and how does it compare to OpenAI's model?

AThe 8B version scored 78.2% on the WebVoyager test, which is very close to the performance of OpenAI's proprietary model o3, which scored 79.3%.

QWhat is the name of the large, open dataset released alongside MolmoWeb, and what does it contain?

AThe dataset is called MolmoWebMix. It contains 36,000 real browsing tasks completed by human volunteers, over 2.2 million screenshot-QA pairs, and automated synthetic data verified by GPT-4o.

QOn which platforms has MolmoWeb been made available, and under what license?

AMolmoWeb has been fully released on Hugging Face and GitHub under the Apache 2.0 license.

İlgili Okumalar

Ethereum Foundation adds SEAL 911 co-founder to board as privacy focus grows

The Ethereum Foundation (EF) has appointed crypto security expert Pascal "pcaversaccio" Caversaccio, co-founder of SEAL 911, to its board of directors. This voluntary one-year appointment aligns with the EF's heightened organizational focus on privacy and security, which it recently deemed "non-negotiable protocol guarantees." Caversaccio is a long-time contributor, a member of the EF's Silviculture Society advisory group, and the author of key documents on Ethereum's cypherpunk ethos and privacy roadmap. The board, which also includes Aya Miyaguchi, Vitalik Buterin, and Patrick Storchenegger, sets the EF's vision and acts as a "security council" to protect its mission. The move underscores the Foundation's commitment to advancing layer-1 privacy, post-quantum security, and Ethereum's self-sovereignty.

cointelegraph21 dk önce

Ethereum Foundation adds SEAL 911 co-founder to board as privacy focus grows

cointelegraph21 dk önce

Fed Drops 'Hawkish Bomb' Late at Night: Interest Rates on Hold, Internal Divisions Unseen in a Decade

The Federal Reserve held interest rates steady at 3.50%-3.75% on July 29, marking the fifth consecutive pause since late 2026. While the decision was widely expected, it revealed a significant internal rift. Three FOMC voting members—Hammack, Kashkari, and Logan—dissented, advocating for a 25-basis-point hike. This level of opposition, the first of its kind since 2016, signals persistent hawkish pressure. Chairman Wash, in a post-meeting press conference, delivered a firm message, emphasizing an unwavering commitment to the 2% inflation target and stating the Fed "would not hesitate to act." He announced a substantial shift away from forward guidance, urging markets to interpret economic data independently. Wash also highlighted AI-related capital spending as a key variable, noting its potential dual impact on boosting productivity while potentially pushing up certain prices. Market reaction was pronounced. Stocks fell sharply, with the Dow dropping over 2% for its worst day in 15 months. Long-term Treasury yields surged, with the 30-year yield hitting its highest since 2007, while the dollar weakened and gold briefly rose above $4,100. The combination of internal policy dissent and a shift toward data-dependent communication introduced significant uncertainty into the market outlook.

marsbit23 dk önce

Fed Drops 'Hawkish Bomb' Late at Night: Interest Rates on Hold, Internal Divisions Unseen in a Decade

marsbit23 dk önce

After R&D Investment Catches Up with the U.S., Is the Sino-U.S. Chip Competition Still Just About Money?

The article examines the significance of recent data showing China's total R&D expenditure, measured by purchasing power parity (PPP), catching up to or slightly surpassing that of the United States in 2024. It argues that while this milestone reflects China's immense capacity to mobilize research resources, the competition in semiconductors now extends far beyond sheer financial input. The analysis highlights key differences: China's R&D is heavily skewed towards experimental development (over 80%), focusing on product engineering and industrialization, whereas the U.S. allocates a proportionally larger share (about 15%) to basic research. Furthermore, leading U.S. semiconductor firms reinvest a significant portion of their substantial global sales revenue into R&D, creating a sustainable commercial innovation cycle that is difficult to replicate. The article emphasizes that semiconductor progress depends on converting R&D into commercially viable products that pass rigorous customer validation and achieve repeat orders, not just on spending levels. It concludes that as China enters the top tier of R&D spenders, the critical challenges shift to improving resource allocation efficiency, fostering long-term basic research, and building effective mechanisms to bridge the gap between laboratory discoveries and reliable, market-ready industrial capabilities.

marsbit1 saat önce

After R&D Investment Catches Up with the U.S., Is the Sino-U.S. Chip Competition Still Just About Money?

marsbit1 saat önce

Walsh: 2% Inflation Target Unwavering, Maintain Independence, Focus on AI Transformation (Full Text Attached)

Fed Chair Warsh emphasized an unwavering commitment to the 2% inflation target, stating there is "no soft target" and that the FOMC is united on this goal. He indicated that if high inflation persists, interest rate hikes would likely be part of the solution. The Fed voted 9-3 to maintain the benchmark rate at 3.5%-3.75%. Warsh announced a significant shift in policy communication, moving away from forward guidance. He noted that market rates have risen between meetings, reflecting a tightening of financial conditions as participants now react more to direct economic data rather than Fed signals. On AI, Warsh highlighted a nearly 20% annual growth rate in high-tech capital expenditures related to artificial intelligence. He acknowledged its potential to boost productivity and supply but also its role in driving up prices for certain sectors like chips and AI infrastructure. The Fed is assessing whether these price pressures will remain sector-specific or spread more broadly. Warsh stressed the Fed's independence, asserting it will not deviate from its mandate due to external pressures. He framed the current decision not as a "pause" but as a period of rigorous review of economic conditions, with future actions to be data-dependent.

marsbit1 saat önce

Walsh: 2% Inflation Target Unwavering, Maintain Independence, Focus on AI Transformation (Full Text Attached)

marsbit1 saat önce

Top 10% of American Households Capture 88% of Wealth, How Is the AI Era Cake Divided?

AI Worsens Wealth Inequality as Top 10% of US Families Garner 88% of Stock Gains (2019-2026) A report from the China Finance 40 Forum highlights that the AI boom is significantly widening wealth inequality in the United States. From 2019 to Q1 2026, wealth from directly held stocks by US households nearly doubled from $29 trillion to approximately $55 trillion, with rapid growth concentrated post-2023, coinciding with the AI-driven stock market surge. The distribution of these gains has been starkly uneven. Between 2022 and Q1 2026, the wealth increase of about $21 trillion was captured almost entirely by the wealthiest families: the top 10% secured roughly 88% ($18.5 trillion), while the bottom 50% received only about 1% ($0.2 trillion). This has contributed to a growing disparity in disposable income shares. The report, referencing economic historian Robert Allen, draws parallels to historical technological shifts like the "Engels' Pause" during the First Industrial Revolution, where worker wages stagnated despite productivity gains. It suggests AI could induce a similar period where capital收益 outpace labor income, exacerbating inequality. Huang Yiping of Peking University identifies four mechanisms through which AI impacts income distribution: capital-bias (reducing labor's income share), task polarization (hollowing out middle-skill jobs), skill-based digital divides, and wealth amplification through assets. He warns that if this trend continues, strong supply growth driven by AI could be undermined by persistently weak consumer demand, threatening sustainable economic growth. To address these challenges, the report proposes a three-pronged strategy: 1) Defensive measures like strengthening social safety nets and antitrust enforcement; 2) Empowering workers through education reform and lifelong learning to collaborate with AI; and 3) Rebalancing via policies such as potential taxes on AI超额收益 and mechanisms for broader sharing of technology's benefits, ensuring AI's红利 are more equitably distributed.

marsbit1 saat önce

Top 10% of American Households Capture 88% of Wealth, How Is the AI Era Cake Divided?

marsbit1 saat önce

İşlemler

Spot