Artículos Relacionados con AI Benchmark

"Agents' Last Exam", Claude Fable 5 Actually Loses to GPT 5.5

Surprisingly, in the newly released "Agents' Last Exam" (ALE) benchmark from UC Berkeley, GPT-5.5 has outperformed the recently launched and highly-regarded Claude Fable 5. ALE tests AI agents on their ability to perform real-world tasks across 55 professional domains—such as 3D modeling in Siemens NX, creating game scenes in Unreal Engine, and visual effects work in Adobe After Effects—by granting them full GUI and command-line access. In the core task completion rate ranking, GPT-5.5 configurations secured the top two spots (24.0% and 23.0%), while Claude Fable 5 with Claude Code came in third (22.0%). Notably, the highest pass rate was only 24%, and the most difficult "Last-Exam" tier saw most top models, including GPT-5.5 and Fable 5, scoring zero. The benchmark also revealed significant cost and efficiency gaps: Fable 5 spent over four times more money than GPT-5.5's most expensive configuration for a slightly lower score, and was much slower. ALE differs from previous knowledge-based benchmarks by evaluating practical "ability to do" rather than static knowledge retrieval. Its tasks are derived from real expert projects, automatically scored, and designed to prevent cheating through a rotating pool of private challenges. The results suggest that high performance on traditional benchmarks does not necessarily translate to proficiency in complex, open-ended real-world work. The study also notes that agents often fail by prematurely declaring tasks complete without proper verification, and that no single model excels uniformly across all diverse domains.

marsbitHace 6 hora(s)