As smart contracts evolve from small experiments into major financial systems managing over $400 billion in assets, security has become increasingly critical.
Unlike traditional software, most blockchain programs cannot be changed after deployment, meaning even minor coding errors can cause permanent financial losses.
To evaluate how artificial intelligence performs in this high-risk environment, researchers from OpenAI, Paradigm, and OtterSec developed EVMbench.
Instead of simple test challenges, it uses 120 real vulnerabilities from 40 blockchain projects, making the evaluation closer to real-world conditions.
Remarking on which, the OpenAI blog post noted,
“We evaluate a range of frontier agents and find that they are capable of discovering and exploiting vulnerabilities end-to-end against live blockchain instances.”
It further added,
“We release code, tasks, and tooling to support continued measurement of these capabilities and future work on security.”
Is AI actually reshaping smart contract security?
While AI greatly improves auditing and bug fixing, it can also exploit system weaknesses. To resolve this, EVMbench helps researchers track these risks.
It also guides responsible AI development for high-value financial systems.
That being said, EVMbench tests AI agents in three stages.
Each stage represents a different level of technical difficulty, reflecting growing security responsibility.
The community appreciates this effort
Appreciating this move, an X user account noted,
“This is a watershed moment for smart contract security. The jump from 31.9% to 72.2% exploit success in just 6 months shows AI agents aren’t just getting better at reading code—they’re mastering the full attack chain.”
Echoing similar sentiments, another user added,
“The 6× jump in exploit success is wild progress, but kinda worrying how fast offensive skills are scaling.”
Recent incident that sent shockwaves
Yet, despite such optimism, something unreal happened soon after OpenAI launched EVMbench. An exploit involving Claude Opus 4.6 raised serious concerns about the risks of “vibe-coded” smart contracts.
In this case, the AI helped write vulnerable Solidity code that incorrectly set the price of the cbETH asset at $1.12 instead of its real value of around $2,200, triggering liquidations and causing losses of nearly $1.78 million.
This shows that trusting AI with critical financial logic without careful human review can turn small mistakes into major losses.
Limitations remain
EVMbench has clear limitations. It includes only 120 curated vulnerabilities and cannot evaluate newly discovered issues.
Detect Mode also produces false positives. While the small number of Patch and Exploit tasks reflects the heavy manual effort needed to create them.
In addition, the sandboxed environment fails to fully represent real-world conditions such as cross-chain activity, timing complexities, and long-term network history.
Needless to say, as blockchain adoption accelerates, its misuse is evolving just as quickly.
Recently, research by Group-IB also showed that the DeadLock ransomware is using Polygon smart contracts to conceal server infrastructure and evade detection.
Together, these developments signal a troubling shift where smart contracts, originally designed to enhance transparency and trust, are increasingly being repurposed as tools for cybercrime.
Final Summary
- Tools like EVMbench help researchers measure AI capabilities in realistic security settings.
- Limited datasets and controlled environments still fail to capture real-world blockchain complexity.
