This time, Claude has truly created a gap in the AI programming rankings!
On the freshly updated MirrorCode leaderboard, Claude Fable 5 has once again topped the chart with an absolute success rate of 64%.
Closely following is GPT-5.6 Sol, with a score only one-third of Claude's!
GPT-5.5, ranked fourth, fared even worse. Not only did it achieve only 10%, but it was also beaten by its own predecessor, GPT-5.4.
What's even more surprising is that when using high-resource languages like Go, Fable 5's solve rate is 64%; when switched to the niche language Ada, its score remains as high as 61%.
It's worth noting that in the open-source world, Python corpus is about 230 times larger than Ada's.
But with Fable 5, the score only dropped by 3 percentage points.
This is interesting.
If the model primarily relied on memorizing syntax and common patterns of popular languages, its performance should have dropped when switched to Ada.
But the current results point to another possibility——
The strongest model has broken free from the pull of specific corpora and has begun learning how to construct a complete software project from scratch.
Stuck at the 100% Pass Line, the 100 Billion Token Ultimate Test
Specifically, the complete MirrorCode contains 25 target programs, covering fields such as Unix tools, interpreters, data queries, bioinformatics, cryptography, and compression tools.
Here, the model is placed in an isolated environment—no internet, no access to the original project source code, and no ability to download third-party dependencies. All it gets is high-level documentation, a portion of visible tests, and access to call the original program repeatedly.
Next, it must continuously feed data to the original program, observe the output to guess the internal logic, and then bit by bit write a new program with consistent behavior.
The latest leaderboard selects 15 Medium and Large targets from this. Each target uses two implementation languages, and each language is run three times.
Moreover, the completion rate for both visible and hidden tests must reach 100% to pass; 99.9% is not acceptable.
If even one edge case is missed, the entire run is still counted as a failure.
To force the model to fill the last few gaps, MirrorCode pushed the single-run budget to 100 billion tokens, allowing continuous runs for up to 7 days.
One of the most expensive tasks mentioned in the paper is even more exaggerated: the model ran continuously for 19 days, with a single-run cost reaching $2,600.
During these 19 days, the model would repeatedly run the original program, compare results, add missing features, and run the tests again. If there was an error, it would investigate the cause; if the output didn't match, it would change assumptions; once a part passed, it would move on to the next gap.
The entire process resembles debugging lasting several days more than a single generation.
The example of 'gotree' is the most straightforward.
This bioinformatics tool originally had about 16,000 lines of Go code and over 40 commands.
Claude Opus 4.7 took 14 hours and $251, passing 2000 out of 2001 tests, achieving a completion rate of 99.95%.
Although it missed one obscure edge case handling date comments, getting blocked by MirrorCode's 100% pass line, it compressed a project estimated to take weeks into just over ten hours——
Epoch estimates that without using AI, a human engineer would need at least 2 to 17 weeks to complete the same task.
In the Corpus Desert, Fable 5 Dropped Only 3 Points
The MirrorCode paper used StarCoder's public training mix as a reference.
Python accounts for about 8%, while Ada is only 0.034%—the former is roughly 230 times larger than the latter.
Of course, we cannot know how much Ada code closed-source models have actually seen. But using the public ecosystem as a reference, the scarcity of Ada is already clear enough.
This language primarily appears in aerospace, defense, and other safety-critical systems. Its community size, number of tutorials, and open-source projects are far from comparable to Python, JavaScript, or Go.
And Fable 5 clearly isn't translating Go code line by line into Ada.
It seems more like it first figures out how the original program works, then switches to another language to recreate the same behavior.
In comparison, other models aren't as stable.
GPT-5.6 Sol dropped from 24% to 19%, GPT-5.4 from 21% to 12%, and GPT-5.5 even fell from 17% to 5%. Once the language changed, the gap was immediately magnified.
Handing Over the Entire Project to AI
Nowadays, Cursor has enabled hundreds of Agents to collaborate for nearly a week, writing over 1 million lines of browser code from scratch, distributed across 1,000 files.
Anthropic had 16 Claude Agents run nearly 2,000 sessions in parallel, eventually building a C compiler with 100,000 lines capable of compiling the Linux 6.9 kernel.
In OpenAI's disclosed Codex personal user samples up to May, 70.2% had submitted at least one task estimated to require over one hour of human work; 25.6% had submitted tasks estimated to take over eight hours.
The units people are handing over to AI are shifting from a piece of code, a bug, to an afternoon's work, a week's work, or even an entire project.
The 64% from MirrorCode is precisely a quantification of this kind of "project-level delegation."
It shows that as long as the goal is clear enough and the results can be automatically validated, frontier models can already independently complete a portion of medium to large-scale software.
For everyone who is starting to hand work over to AI, the change is already imminent——
Previously, you needed to watch it write every line of code. Next, you're more likely to only check if it has gone off track at key junctures.
Code will become increasingly cheap.
And those who can articulate problems clearly and validate results effectively will become increasingly valuable.
Reference: https://epoch.ai/MirrorCode
This article is from the WeChat public account "Xinzhiyuan" (New Wisdom Source), author: ASI Apocalypse; editor: Moses
