Without further ado, let's start the daily slump (doge)!
Just moments ago, the great Karpathy announced that "we" at Anthropic have started using a brand new way to push large models to their limits—
"The Lord of the Rings."
According to Karpathy, this "Lord of the Rings benchmark" is taking over from the once-popular "Pelican Riding a Bicycle" SVG test.
Specifically, Karpathy directly fed the beginning of "The Lord of the Rings" to Opus 5, instructing the model to use Three.js to craft an entire "Middle-earth" world on the spot.
As for the final effect... well, it somewhat resembles domestic 3D animations from the late 1990s and early 2000s: very rough and quite abstract.
But objectively speaking, if you look closely for a while and happen to be familiar with the "Lord of the Rings" filming location Hobbiton in New Zealand, you can indeed catch a hint of its flavor~
However, this seemingly unpolished creation genuinely cost Opus 5: 1 million tokens, 2 hours, and 5,500 lines of code.
Of course, it's not just the Claude model shining solo on this stage.
The most hilarious part is probably the new offering from netizens—a DeepSeek V4 Flash version.
It's a complete "Chinese people can fly" situation, with all characters floating in the air.
Faced with these currently obvious goofs, Karpathy was quite fair.
He pointed out that Opus 5 still cannot truly "enter" the world it generates; it can only continuously take screenshots at different time points and slowly check where things went wrong.
And this precisely exposes a significant shortcoming of current large models:
They can already write code, build scenes, and generate games, but they still cannot truly understand videos, nor can they actually play the games they create.
Two Hours, Crafting Middle-earth by Hand
Let's first look at how this "Lord of the Rings" test was conducted.
If taken literally from Karpathy's tweet, the main prompt input for Opus 5 should be the beginning of Chapter One, "A Long-expected Party," from "The Lord of the Rings" novel.
When Mr. Bilbo Baggins of Bag End announced that he would shortly be celebrating his eleventy-first birthday with a party of special magnificence, there was much talk and excitement in Hobbiton...
Interested folks can try it themselves.
In addition, Karpathy also specified Three.js in the prompt—a JavaScript library for building 3D scenes with code.
Thus, the task for Opus 5 was to first comprehend the text at the beginning of "The Lord of the Rings," and then translate it into a 3D world that could run in real-time in a browser.
Throughout the process, Opus 5 needed to use polygons to piece together characters, buildings, and props, place them one by one into x, y, z coordinate systems, and then arrange cameras, lighting, and animations.
When characters should move, where the camera should turn, how lighting should change, and how objects in the scene should interact—all needed to be defined by the model using code.
Although the final visuals aren't exactly refined, and issues like clipping and floating frequently occur—for example, characters' bodies and heads separating... but at least the entire scene was genuinely built.
It's important to note that this is not the same as video models directly generating pixel frames.
Three.js first needs to build characters, objects, and scenes as 3D objects, then calculate their positions, angles, and motion states in real-time based on the code.
Therefore, the demo we see is not a regular AI-generated video, but a recording of a running 3D web scene.
However, Karpathy also mentioned in the comments that procedural 3D and video generation are not an either-or choice.
A netizen suggested that this rough Three.js recording could be fed to Sora as a reference video, letting the video model render it again with higher visual quality.
Karpathy quickly agreed.
According to his vision, procedural code could handle storyboarding and control, first finalizing the skeleton: where characters stand, how the camera moves, and how the plot progresses.
Then, the recording could be passed to a Video-to-Video model to add textures, lighting, shadows, and details, enhancing the overall "appeal" of the Middle-earth world.
As for audio, Opus 5 didn't handle it all this time.
Karpathy stated that, due to personal requirements for sound quality, he ultimately used ElevenLabs.
Thus, that "Lord of the Rings" demo with visuals, camera work, and narration at the beginning came into being.
Karpathy has also open-sourced the entire project; the specific link can be found at the end.
Netizens Are More Interesting Than Karpathy
After Karpathy released the demo, many netizens also rushed to test it out.
Looking at the overall response, one can only say:
This bunch of netizens is more imaginative than Karpathy.
Someone used Claude to launch an 'Earth Online' project, aiming to build the entire Earth bit by bit with a group of AI Agents.
Currently, the Agents haven't finished the whole Earth, having only built a low-poly-style San Francisco waterfront area. But from the available visuals, the building proportions, character scales, and overall style remain quite consistent.
This effect is somewhat like those Lego-world animations. The art style isn't complex, but characters, scenes, and actions can operate stably within the same world.
Continuing in this direction, simple-styled animations and lightweight games are already showing a hint of AI-native potential.
Another netizen used Fable 5 and GPT-5.6 Sol to build a 3D digital model of New York City, incorporating real-time data into it.
The model not only had to position New York's streets and buildings correctly but also maintain spatial relationships between different areas. The author thus proposed that this task of generating virtual worlds could perhaps become a new benchmark for spatial reasoning.
Even more exaggerated examples followed.
One netizen couldn't get tickets to Kanye West's concert, so they simply used AI to host a private concert for themselves in the browser.
The entire project was still built with Three.js. It consisted of a single HTML file, without calling any pre-made 3D models; the stage, characters, and lighting were all generated by code.
The concert prepared 14 songs in total, each with independent lighting design and a dedicated set of spherical stage visuals.
Regular users could listen to clips; after connecting a Spotify Premium account, they could play full songs and the entire show.
Couldn't get tickets? Just generate a private show for yourself. Incredible!
And there's more!!!
Karpathy also envisioned in the original post that gameplay could be added to these 3D worlds later, allowing players to enter as observers, NPCs, or even story characters.
Before Karpathy could arrange a game version, netizens had already made one.
This project also used Opus 5 and Three.js. It could not only run and be interactive but also had audio added.
More 3D design cases kept appearing. From building proportions and spatial layouts to scene style, Opus 5 could already maintain relatively stable consistency across fairly large-scale projects.
There are many similar examples; we won't showcase them all here.
Objectively speaking, these works still have some distance from truly mature games.
Whether the dazzling array of gameplay is fun, if it runs stably over long periods, and if players would genuinely stay for 15 minutes—all these questions currently lack answers.
But the barrier to creating real-time 3D content and playable prototypes has indeed been significantly lowered.
Moreover, having a new way to test model capabilities that is also sufficiently fun and interesting—isn't that a beautiful thing?
The Pelican Has Ridden Far Enough
So, why suddenly ask large models to generate "The Lord of the Rings"?
This goes back to the wildly popular "Pelican Riding a Bicycle" test from a few years ago.
This problem originated from developer Simon Willison, with only one sentence as the requirement:
Generate an SVG image of a pelican riding a bicycle.
Although this problem seems simple, it actually tests models quite significantly.
Because an SVG looks like an image, but its underlying structure is a string of code.
The model not only needs to know what a pelican and a bicycle look like separately but also must break them down into lines, circles, and polygons, then arrange the positional relationships of each component using coordinates.
More crucially, pelicans and bicycles themselves are not a natural fit.
The bicycle's frame, tires, and pedals must maintain correct geometric structures; the pelican has a large beak, short legs, and a physique that doesn't look at all suited for pedaling.
Combined, whether the model truly understands spatial relationships becomes almost immediately apparent:
Are the wheels crooked? Are the feet stepping on the pedals? Is the bird actually riding, or has it been dismembered by the bike frame?
Take a look at these demos from late 2024 above~
For this reason, "Pelican Riding a Bicycle" once became a classic folk test for observing large models' spatial understanding, object composition, and code generation abilities.
But as models grew stronger, this pelican's ride is nearing its end.
Just look at the performance of DeepSeek R1 and DeepSeek V4 below. Nowadays, models can handle this problem quite decently.
More importantly, a single SVG can only test a model's one-time output.
It cannot measure whether a model can plan a complex project, work continuously for hours, and repeatedly check and correct its own mistakes across thousands of lines of code.
Thus, Karpathy replaced a small "draw a picture" problem with a major "build a world" project—
The pelican can dismount now; Middle-earth officially takes over.
Karpathy's Pelican
Soon, news that Karpathy was preparing to change the "Pelican test" problem also spread to various communities.
High-rated comments on Hacker News argued that although the visual quality of these demos is generally low, this precisely indicates we need a new test more challenging than generating a single image.
The new benchmark shouldn't just see if the model can draw the picture correctly; it should also examine whether it can comprehend a world.
After all, "Pelican Riding a Bicycle" has been used for too long.
Models continuously optimize for this type of task on leaderboards, making the differences between them harder to discern.
In contrast, generating a complete 3D world requires the model to understand spatial relationships between characters and objects, handle camera, motion, and scene changes—not simply call a video generation model to spit out a sequence of frames.
Of course, there were opposing opinions as well.
The Pelican test is sufficiently concise, low-cost, and its results are easy to compare.
Consuming so many Tokens to generate an entire 3D world just to test a model once seems a bit like setting off fireworks with computing power.
Simultaneously, whether Three.js itself can measure a large model's comprehensive abilities was also questioned.
Some argued that such demos can at most prove that Anthropic has trained well on Three.js code, not that the model truly understands space and the physical world.
But soon, netizens rebutted:
Transforming an abstract, ambiguous literary text into a 3D animation requires the model to simultaneously handle spatial relationships, physical laws, everyday objects, as well as mathematical problems in 3D transformations and computer graphics.
If this is still considered merely "knowing how to write Three.js," that somewhat underestimates these 5,500 lines of code.
Another netizen raised an even more open question:
Is so-called "spatial reasoning" truly different from the reasoning large models typically perform when processing text and code?
One view holds that whether a scene holds up depends on whether the model understands spatial relationships like "front/back, inside/outside, near/far, occlusion," as well as the distance, angles, and relative sizes of objects from different viewpoints.
But another view argues that whether the model processes "beside a stone" or "inside an array," it might be doing the same thing: generating Tokens one by one based on context, completing reasoning in the process.
If so, then what Opus 5 demonstrates is not just a suddenly emerging "spatial ability."
A more likely scenario is that the general reasoning ability originally used by large language models to understand text and code has begun naturally extending into the three-dimensional world.
From drawing a pelican to building Middle-earth, the problem seems to have changed, but what's being tested behind the scenes might always be the same question:
Can the model transform its understanding of the world into a structure that can truly run.
And finally, perhaps there's an even more outrageous question—
If general large models can already write code to build 3D worlds themselves, then call APIs like Sora and ElevenLabs to complete visuals and audio, how much need is there left for users to personally open a specialized video generation product and input a Prompt?
Reference Links
[1]https://karpathy.ai/lotr-movie/
[2]https://simonwillison.net/2025/Jun/6/six-months-in-llms/
[3] https://x.com/wizardbrainz/status/2083012159341203708
[4]https://x.com/aniketjart/status/2083645765097033845
[5]https://x.com/davidfromkansas/status/2075691129899528254
[6]https://x.com/MindaugasLT/status/2083488027343470939
This article is from the WeChat public account "QbitAI," author: henry
