The former OpenAI chief scientist, Ilya, is a classic example of "the legend has left the arena, but his stories still linger everywhere."
Having been instrumental in ushering in the era of deep learning, it has been over two years since he left OpenAI. The SSI he founded himself doesn't release models, doesn't launch products, almost disappearing from the playing field.
Yet, from time to time, someone in the AI community still asks the same question: What is Ilya doing?
Now, suddenly, there seems to be an answer to this question.
Just now, a netizen named "Three Strawberries" on X爆料ed that SSI is exploring a small inference engine based on TTT (Test-Time Training).
According to the爆料, this model learns "how to learn" from specially curated data and then updates part of its weights during the problem-solving process.
Even if the model is not large in scale, it might be able to compete with models trained on much larger scales.
Furthermore, the爆料er indicated in the comments that SSI's current version is already ready, and the team is working to scale the next-generation version 10 times larger.
And, even more crucially, this might not just be an empty promise of "potential."
According to the爆料, the current version could be revealed as early as this August, initially available to a limited number of users.
That means, if the爆料 is true. The first blade forged by SSI after two years of secrecy might truly be unsheathed.
After the news broke, many netizens already sensed the shift in the wind.
Some exclaimed that this is starting to resemble true intelligence;
Others judged that as long as this approach can scale smoothly, other companies would find it difficult to catch up with SSI.
But just as everyone was preparing to welcome "Ilya's triumphant return," there was more news:
It might not be released this month after all.
Truth and falsehood, intertwined~
This time, will Ilya's sword finally be unsheathed?
A Sword that Changes Its Moves Mid-Strike
To be honest, since Ilya left OpenAI to start his own venture, rumors about SSI have been flying everywhere.
Some questioned Ilya's real contributions in the history of deep learning, suggesting the true father of GPT was actually Alec Radford;
Others said he, like Liang Wenfeng, had turned to study quantization.
But the reason this爆料 exploded so quickly isn't just because the name "Ilya" carries its own weight.
It's because it actually aligns with several publicly available clues, at least three upon梳理ing:
First, continuous learning after deployment, which highly aligns with Ilya's views expressed in a Dwarkesh Patel podcast in November last year.
At the time, Ilya stated: The terms "AGI" and "pre-training" have misled the industry.
Because humans are not born with general intelligence capable of all tasks; what humans truly rely on is continuous learning.
The superintelligence he envisioned is more like an extremely intelligent, curious 15-year-old.
It doesn't know that much, but it can learn programming, medicine, and various jobs after deployment, continuing to grow through trial and error.
In Ilya's view, what SSI aims to build is not a machine that knows everything at the factory exit (after pre-training ends), but a mind that can learn any task.
This goal carries a distinct meta-learning flavor: what the model needs to master is not just a specific skill, but the method of learning new skills.
Ilya didn't mention TTT at the time, but they point to the same question:
Can the model continue to change itself after deployment?
The second clue comes from NVIDIA.
On July 27 this year, NVIDIA and SSI announced a long-term strategic partnership, including investing in SSI and providing it with the next-generation Vera Rubin systems.
SSI's computing power will increase by an order of magnitude, roughly 10 times, and according to Reuters爆料, the investment amount is as high as $5 billion.
Crucially, money and GPUs aren't the most important part.
NVIDIA specifically wrote in the announcement that SSI has been quietly advancing a new research direction over the past two years.
After NVIDIA gained rare access to this tightly guarded research, they decided to push it into the next phase.
In other words, Jensen has seen the cards. As for what the cards are, he didn't say.
Ilya's own statement was equally intriguing: It's time to scale SSI!
Well... this scaling is quite suggestive.
Ilya, what have your eyes seen?
The third clue is the progress of TTT itself over the past two years.
Recently, more and more researchers have begun to look beyond the route of "constantly expanding context" and search for a paradigm closer to continuous learning.
The reason is simple. Because humans don't learn by making cheat sheets longer and longer, nor do they, like the protagonist in *Memento*, tattoo every encountered event on themselves.
After experiencing something, the brain adjusts its internal connections. The next time a similar problem is encountered, the person is no longer the same as before. This is the truly interesting aspect of the TTT approach.
Simply put, TTT (Test-Time Training) means training at test time. It prevents the model from completely separating "training" and "inference." Instead, while solving real problems, the model continues to train itself on the immediate data, then adjusts its internal state, or even directly updates some parameters, based on the error between the prediction and the target.
In the爆料, "Three Strawberries" also specifically mentioned a related paper—
End-to-End Test-Time Training for Long Context.
In short, this paper reframes the long-context problem as a continuous learning problem.
Traditional long-context models save previous text in a context window and look back for needed information each time they generate new content.
The TTT-E2E proposed in the paper treats the entire context as training data, allowing the model to continue doing next-token prediction while reading, compressing the information it reads into its own weights.
Thus, context no longer remains just a cheat sheet to refer back to, but begins to become learning material that genuinely changes the model itself.
It's worth adding that TTT is fundamentally different from the recently popular Agent Harness and Context Engineering.
The latter two rely on external context to build scaffolding, allowing the model to see more information and use more tools, but the model itself does not change.
TTT goes a step further. It doesn't just let the model see more or use more; it allows the model to directly change itself during the problem-solving process.
So, the question returns.
If TTT isn't new and is so amazing, why hasn't it been widely adopted sooner? The reason is not hard to guess:
Because once a model can modify itself after deployment, it might learn wrong things, become biased, or even forget its original capabilities.
In 2016, Microsoft's chatbot Tay was taken offline within 24 hours after being诱导ed by netizens to output offensive content.
Tay and TTT are not the same technology, but they exposed the same challenge early on:
Safety.
So, naturally, a netizen raised this question, and then Three Strawberries indicated that Ilya had already solved it.
Of course, this statement is far from evidence and even sounds like a flimsy rumor.
But ironically, it circles back to the S in SSI (Safe Superintelligence):
Safe.
If Ilya truly wants to build a superintelligence that continuously grows after deployment, then "teaching it how to learn" only solves half the problem.
The other half, perhaps even harder, is: teaching it what not to learn.
Why Is Everyone Waiting for Him
Speaking of this, there's one last question:
Why does an anonymous爆料 of unverified authenticity excite so many people?
Simply because the person behind the closed doors is Ilya Sutskever.
He is an OpenAI co-founder, former chief scientist, and one of the most influential researchers in deep learning over the past decade.
From AlexNet, Sequence-to-Sequence, AlphaGo to the GPT series, his name can be seen.
NVIDIA also specifically mentioned in this collaboration announcement that Ilya later led the research that ultimately led to OpenAI's o1 and other reasoning models.
In other words, his most frightening attribute has never been releasing many models.
But rather, in his career, several times before a technical path became consensus, he was already standing on that path.
It is precisely for this reason that after leaving OpenAI in May 2024, what Ilya's next move would be has remained one of the biggest mysteries in the AI community.
And SSI, established just one month after his departure, ingrained mystery into its very bones.
Besides its goal of Safe Superintelligence, for the next two years, SSI didn't release models, launch products, or publish papers. The outside world didn't even know what they were specifically researching.
Even when Ilya occasionally appeared, he only spoke repeatedly about generalization, continuous learning, and that the pre-training era might be nearing its end.
Until this July, when NVIDIA decided to double down after seeing SSI's highly confidential research and helped boost its computing power by an order of magnitude, did SSI return to the public eye.
Therefore, looking at today's TTT爆料 again, it tastes different.
Ilya has been saying that a true superintelligence should be able to continue learning after deployment;
Now someone爆料s that SSI is working on a model that continues to update itself during inference.
Of course, there's still no solid proof. But if these two threads could eventually align—
Then what Ilya has been forging in seclusion for two years might truly not be just another bigger GPT.
But a sword that changes its own moves mid-strike.
Get ready!
Reference Links[1]https://x.com/i/trending/2087551605553901958[2]https://www.youtube.com/watch?v=aR20FWCCjAs
This article is from the WeChat public account "Qubit," author: Focus on Frontier Technology
