Recently, we have reported on many startups and research findings targeting "continual learning," such as "Mind Lab Continuously Releases Latest Progress on LoRA, New Paradigm for "Continual Learning" of Large Models Emerges," "Farewell to the Shackles of KV Cache, Compressing Long Context into Weights, Is Continual Learning for Large Models Promising?," "ICML 2026 | Breaking Limits! HKU Proposes First Continual Learning Architecture Adaptable to 300+ Tasks, Solving the Forgetting Problem," "Harness That Can Make DeepSeek Self-Evolve! LlamaFactory Author Open Sources New Tool: Automatically Creating Agents for 0.2 RMB," "When 'Getting Bigger' Is No Longer the Only Path, Another Domestic Model Is Open-Sourced"......
Yes, quite dense, "continual learning" has also become one of the most frequently encountered keywords for us. This also signals a repeatedly mentioned judgment behind it.
In October 2025, Andrej Karpathy said on Dwarkesh Patel's podcast: Current large models "have no continual learning. You can't tell it something and expect it to remember." He believes fixing these cognitive deficits will probably take another decade. Two months later, he placed this statement into a larger picture in his annual review: The leap in model capability in 2025 mainly came from verified reward reinforcement learning (RLVR), but within the entire LLM tech stack, memory, multimodal perception, continual learning, and the ability to operate computers remain significant shortcomings. "We have prototypes, but we don't yet have agents that can act as colleagues."
Continual Learning (also called Lifelong Learning) thus became one of the hottest concepts in the past year.
It refers to the model's ability, after deployment, to continuously absorb knowledge from new tasks, new knowledge, and new experiences like a human, without forgetting what it has previously learned.
This sounds obvious and natural, but is extremely difficult to achieve, difficult enough to be one of the hardest bones to crack on the road to "AI colleagues." And the series of reports above show that this path is no longer just one direction, but several diverging routes being advanced simultaneously.
Surrounding how to make models "learn while using," academia and industry have diverged into several distinct technical routes over the past year. Some attach memory externally to the model, some continuously rewrite weights, some simply retrain from scratch, and a newer batch of ideas attempts to redefine the very concept of "learning" itself. This article attempts to lay out these directions one by one, clarifying what each path is betting on and where the challenges lie.
First, Clarify: The Difficulty Isn't "Learning," It's "Not Forgetting"
The core obstacle of continual learning has a specific name: catastrophic forgetting. A neural network's knowledge is stored in billions of weights. When you use new data to fine-tune the model and update these weights, while the model learns the new task, it often overwrites the parameters carrying old capabilities, leading to a sharp performance drop on previously proficient tasks.
Diagram of catastrophic forgetting. When an artificial deep neural network is trained sequentially on two tasks, it rapidly and completely forgets the first task while training on the second.
This phenomenon was heavily studied in the era of small models, but new troubles emerged with LLMs. The TRACE benchmark specifically for evaluating continual learning in LLMs found that continuously fine-tuning an already aligned model not only makes it forget old tasks but also impairs general capabilities and instruction-following ability. In other words, the cost of teaching a model something new may be it becoming overall dumber and less obedient.
Precisely because directly modifying weights carries such high risks, "continual learning" has split into several schools. Their fundamental divergence actually lies in making different trade-offs on the two questions: "modify weights or not" and "where to store new knowledge."
Attaching Memory Outside the Model
The most straightforward and fastest-to-implement idea is: simply don't touch the model weights, store new knowledge in an external database, and retrieve it into the context when needed. This route evolved from RAG (Retrieval-Augmented Generation) and has now grown into a specialized field: Agent Memory.
Representative work is MemGPT (the company behind it is now renamed Letta). It analogizes LLM context management to operating system memory management, distinguishing between limited "working context" and larger "external storage," allowing the model to decide what should be brought into context and what should be archived back, like an OS scheduling memory.
Following this idea, a batch of systems have different emphases: Mem0 focuses on production-grade, scalable long-term memory storage and retrieval; Zep adds a temporal knowledge graph on top of retrieval for cross-session temporal reasoning; A-MEM borrows from the Zettelkasten note-taking method, adding structured tags to each memory and automatically linking them to related old entries, making retrieval more context-aware.
Karpathy himself also bets on this direction. He repeatedly emphasizes that the future needs not "larger hard drive"-type memory, but a lean "cognitive core" (he estimates a couple of billion parameters would suffice) coupled with a set of self-compounding structured external memories.
Based on this idea, he released a pattern called "LLM Wiki" on GitHub: Instead of using RAG to fetch raw text chunks for each query, have the agent actively compile materials into a continuously updated, interlinked knowledge base, and then query that.
Karpathy's LLM Wiki documentation has garnered nearly 45,000 stars and has been forked over 9,000 times.
Letta itself has elevated this into the proposition of "continual learning in token space." They point out that today's default practice is "append first, then summarize," meaning raw experience is continuously piled up until the context overflows, then compressed into a summary. This has two flaws: Appending pushes all representational work to inference time, requiring reprocessing raw logs during each forward pass; and summarization is lossy and abrupt, with important details disappearing without warning. Letta's bet is that memories learned in token space in the future will be more valuable than the model weights themselves.
The strength of this direction is safety, controllability, interpretability—errors can be deleted anytime; the weakness is it doesn't truly "internalize" knowledge into the model. It always relies on retrieval and the narrow gate of context. Once the memory library expands, retrieval accuracy and cost become bottlenecks.
Letting Context Itself Evolve into a "Playbook"
Taking one step further from external memory is a more refined class of ideas: don't modify weights, but let the model's input context itself continuously evolve. This is called Context Engineering.
In October 2025, ACE (Agentic Context Engineering) proposed by Stanford, SambaNova, and UC Berkeley is a representative. It treats context as a constantly growing "playbook," maintained through three specialized roles: Generator is responsible for generating reasoning traces, Reflector extracts specific lessons from successes and failures, and Curator organizes these lessons into structured incremental updates, merging them into the playbook.
ACE aims to solve two common issues with similar methods. One is "brevity bias": when an LLM repeatedly rewrites the context, it tends to compress and lose domain-specific details. The second is "context collapse": repeated rewriting leads to gradual loss of details.
ACE avoids these two points by using incremental small changes (delta updates) rather than rewriting entire sections. According to the paper data, ACE improves performance by 10.6% on agent tasks and 8.6% on financial reasoning compared to baselines, while reducing adaptation latency by about 86.9%; on the AppWorld leaderboard, using the smaller open-source model DeepSeek -V3.1 with ACE achieved an average score comparable to the production-grade agent IBM CUGA based on GPT-4.1.
It is worth noting that ACE emphasizes it can work without labeled supervision; it relies on feedback signals naturally generated during execution (e.g., code runs successfully or throws an error) to guide reflection and organization. This connects it to the larger narrative of "agents learning from their own experience."
Continual Post-training: Modifying Weights, But Intelligently
External memory and context engineering avoid the risks of modifying weights, but also avoid true knowledge internalization. Another group of researchers believes that, in the long run, some knowledge and skills ultimately need to be written into parameters to be truly useful. This is Continual Post-training, mainly divided into stages like continual instruction fine-tuning and continual preference alignment.
John Schulman from Thinking Machines gave a hierarchical view: He analogizes learning to categories in psychology like motor learning, episodic memory, and procedural memory, believing that in-context learning will continue to handle short-range learning tasks, while parameter fine-tuning (including methods like LoRA) will be layered on top, especially suitable for tasks requiring larger capacity and true knowledge absorption—when the time span becomes long and in-context learning becomes insufficient, parameter fine-tuning wins.
The biggest enemy of this route remains catastrophic forgetting, and an interesting recent solution comes from Thinking Machines' Tinker.
Tinker is their first product released in October 2025, a fine-tuning API based on LoRA, abstracting away the complexity of distributed training and only exposing low-level primitives like forward_backward and optim_step, allowing researchers to focus on data and algorithms.
Regarding continual learning, they mainly promote a recipe called Self-Distillation Fine-Tuning (SDFT): The core insight is that ordinary supervised fine-tuning is "off-policy," forcing the model to imitate tokens it would never generate itself, thus eroding old capabilities every time it learns a new skill; SDFT lets the model act as its own teacher, learning new skills from demonstrations without forgetting old ones. A startup called Trajectory has already adopted Tinker as the core infrastructure for its continual learning platform.
Incidentally, using LoRA instead of full fine-tuning in Tinker has another practical benefit: multiple fine-tuning tasks can share the same compute pool, spreading the cost. This also explains why "continual learning as a service" is becoming a business: turning frequent model updates into an on-demand API is precisely the direction industry is currently attempting.
Re-pre-training and Continual Pre-training
If post-training is modifying the "surface layer" of the model, then Continual Pre-training (CPT) returns to the very foundation, continuing to pre-train the model with new corpora to adapt it to new domains, new languages, or knowledge distributions that drift over time. Compared to mixing old and new data and retraining from scratch, CPT is built upon an existing model, making it more cost-effective computationally.
Its typical applications are in three areas: knowledge drifting over time, cross-linguistic expansion, and cross-domain adaptation. But the cost is equally clear: multiple empirical studies repeatedly show that continual pre-training is computationally expensive and prone to triggering catastrophic forgetting of already learned knowledge. Consequently, academia has been searching for "replay-free, task-annotation-free" continual pre-training methods, attempting to achieve non-forgetting at LLM scale.
For the vast majority of companies, the cost of re-pre-training a cutting-edge model from scratch is unrealistically high. This is precisely the origin of Karpathy's statement that "large models are not suitable for frequent retraining." Therefore, strictly speaking, "re-pre-training" is more of an option within frontier labs, while continual pre-training is a more feasible compromise.
Newer Ideas: Letting Models Learn to Self-Modify
The previous four directions more or less revolve within the binary framework of "external vs. modifying weights." However, a newer batch of work emerging in the past year attempts to jump out of this framework and redefine learning itself.
One line of thought is to let the model generate its own training data and decide how to update itself.
MIT's SEAL (Self-Adapting Language Models) is typical. Given a new input, the model generates a "self-edit"; this is a natural language instruction specifying how to restructure information, what hyperparameters to use for updating weights, even what tools to call for data augmentation; then the model fine-tunes itself accordingly, forming persistent weight updates. The matter of "what kind of self-edit is effective" is trained by an outer reinforcement learning loop, with the reward signal being the performance of the updated model on downstream tasks.
The NeurIPS 2025 version further demonstrates that this self-adaptive capability strengthens as model scale increases and, with the help of reinforcement learning, mitigates forgetting. Its authors envision a model that can decide mid-inference "whether to learn now," distilling one-time chain-of-thought into permanent capability.
Another more radical line of thought comes from Google's Nested Learning, published at NeurIPS 2025. Its core is to reinterpret a model as a set of nested, multi-level optimization problems, each with different "context streams" and different update frequencies—architecture and optimization algorithms are unified under this perspective as different levels of the same thing.
As a proof of concept, they created a self-modifying architecture called Hope, extending the Titans long-term memory architecture in two aspects: unlimited nested learning levels and a "Continuous Memory System" (CMS). Simply put, no longer just a short-term/long-term memory dichotomy, but a whole spectrum of memory modules updating across different time scales.
In experiments, Hope outperformed standard Transformers and modern recurrent models in language modeling, long-context reasoning, and continual learning tasks. Interestingly, subsequent work introduced a "sleep" phase to this type of architecture—like humans consolidating memories during sleep, letting the model allocate compute between active sessions to distill useful abstractions into more permanent parameter memory.
At an even higher level of abstraction is the "Era of Experience" narrative proposed by David Silver and Richard Sutton.
Their judgment is: In key fields like mathematics, code, and science, the knowledge that can be extracted from human data is approaching its limit. To continue advancing, AI needs to continuously learn from the experience it generates through interaction with the environment, forming a self-supplying data feedback loop. They view DeepMind's 2024 AlphaProof (winning an IMO medal through "continual interaction with formal proof systems") as the beginning of this era. This narrative binds continual learning together with reinforcement learning and agent autonomous exploration, also leaving an unresolved question: Who designs the reward functions that turn raw signals into useful guidance?
Additionally, there is a neighboring direction easily confused with but having different goals from continual learning: Knowledge Editing. Represented by ROME and MEMIT, it achieves precise single or batch fact updates by locating and modifying the MLP layers storing specific facts, without requiring full retraining. But its goal is actually different from continual learning—knowledge editing aims for precise coverage of a fact. In continuous, lifelong editing scenarios, repeated parameter modifications interfere with each other, gradually accumulating "toxicity" causing model collapse. This has also spurred methods like WISE and AlphaEdit specifically designed to handle sequential editing.
Conclusion
Laying these directions side by side reveals they are essentially arranged along an axis of "where knowledge resides."
The outermost is external memory and context engineering, where knowledge resides entirely in token space outside the model, safe and controllable but not truly internalized; the middle is continual post-training and pre-training, where knowledge is written into weights, offering higher capacity ceilings but facing catastrophic forgetting directly; the innermost, most cutting-edge area includes the new ideas of letting models self-modify and self-evolve, attempting to make the very act of "learning" part of the model's capability.
A pragmatic observation is: these directions are likely not mutually exclusive but will work together in layers. Schulman's hierarchical learning view and Karpathy's "cognitive core + external memory" conception essentially say the same thing: short-range, volatile knowledge is handled by context and external memory; long-range, need-to-settle capabilities are handled by parameter fine-tuning; each performs its own function. The facts that ACE can evolve without supervision using execution feedback and SEAL uses reinforcement learning to optimize self-edits both hint at a common trend: future continual learning will likely involve models taking the lead in "what to learn and how to learn."
So, returning to Karpathy's "another decade" judgment, is continual learning solved yet? The answer is probably—not yet, but it's no longer a blank slate. The core obstacle of catastrophic forgetting has not been completely overcome to this day; methods like SDFT and Nested Learning are still "mitigating" rather than "eliminating" it. Can better context management plus fine-tuning purely solve continual learning, or is a completely new idea needed? Schulman himself admits this more fundamental question remains unanswered.
What can be determined is that during 2025-2026, continual learning transformed from a repeatedly mentioned "missing capability" into an active battlefield with genuinely diverging routes and startups beginning to develop products. Which route can first turn "AI colleagues" from prototypes into reality is worth keeping an eye on.
Reference Links
https://www.lesswrong.com/posts/qBsj6HswdmP6ahaGB/andrej-karpathy-on-llm-cognitive-deficits
https://karpathy.bearblog.dev/year-in-review-2025/
https://www.letta.com/blog/continual-learning/
https://arxiv.org/abs/2510.04618
https://arxiv.org/abs/2402.01364
https://thinkingmachines.ai/tinker/
https://arxiv.org/pdf/2604.05096
https://arxiv.org/pdf/2606.03979
This article is from the WeChat public account "Machine Heart" (ID:almosthuman2014), author: The Continually Learning Machine Heart, editor: Panda
