The AI community never runs short of gossip.
This time, the protagonist is Giambattista Parascandolo, an important researcher in the direction of reasoning models at OpenAI.
In 2020, he went to MIT to interview for a professorship, giving a presentation on using GPT for reasoning. As a result, most professors on the interview committee dismissed this direction as "nonsense."
https://x.com/turingbook/status/2084003612687249836?s=20
This guy went all out, directly putting this experience on his personal homepage and attaching the description and slide link from that year's presentation.
https://sites.google.com/view/giambattista-parascandolo/home
What was in that report dismissed as "nonsense"?
The MIT website shows that the topic of this presentation was how to enable artificial neural networks to break through the training distribution and achieve generalization and planning capabilities closer to humans.
Parascandolo believed that humans can recombine existing knowledge, identify key invariants, build abstract models, and perform long-horizon planning. Artificial neural networks still have much room for improvement in these aspects.
At the end of the presentation, he proposed three future research directions: open-ended reasoning in neural networks, unexplored degrees of freedom in artificial neural networks, and using language as a reasoning substrate in reinforcement learning to improve sample efficiency.
The most crucial concept among these was "open-ended reasoning."
Parascandolo defined it as: the model can invest more time and computation to continuously refine its answer. The harder the problem, the more steps the model should think, and the extra computation should translate into better results.
This already sounds very much like today's inference-time compute scaling.
At that time, standard Transformers had a fixed network depth, and the forward compute each token experienced was basically predetermined. However, problem difficulty has no stable relationship with input length. A problem could be very long but have a simple answer. Another problem might be a single sentence but require multiple rounds of decomposition and verification.
RNNs seemed more suitable for such tasks. They could run repeatedly and theoretically obtain arbitrary amounts of thinking time. However, the curve Parascandolo showed in his PPT indicated that RNNs typically performed best only around the number of reasoning steps they had seen during training. Continuing to increase the number of iterations could actually decrease accuracy.
This meant that increasing compute was only the first step; the model also needed to learn how to utilize this compute.
At the time, the most effective multi-step planning heavily relied on model-predictive control and Monte Carlo tree search. Neural networks were responsible for predicting the environment or evaluating value, while external search algorithms were responsible for expanding future paths. Parascandolo hoped to further integrate long-horizon reasoning capabilities into neural networks themselves.
His second vision was to make language the substrate for reasoning.
Parascandolo used the classic game *Montezuma's Revenge* as an example. A reinforcement learning Agent trained from scratch needed to try a huge number of state and action combinations. Many correct actions themselves were not complex; what the Agent truly lacked was a judgment of "which behaviors are more reasonable."
GPT had already absorbed vast amounts of world knowledge from text. Language could help models describe environments, understand goals, decompose tasks, and generate high-level plans, also significantly narrowing the search space.
Today, this line of thinking easily brings to mind chain-of-thought, language planning, and Agent workflows.
The third direction Parascandolo proposed was that AI systems could reset tasks, return to any remembered state, construct counterfactual scenarios, and even adjust time, gravity, and observations within the simulator. Systems could also directly read, copy, and modify their own activation values and neural network weights.
This was equivalent to incorporating the learning process itself into the Agent's operational space. It could engage in deliberate practice, generate special training scenarios, transfer existing knowledge, and relearn from failure trajectories.
The PPT from five years ago almost wrote today's reasoning model roadmap.
We also dug up a blog post he wrote in June 2021, titled "Backprop, Evolution, and the ‘Two Dogs’ Fallacy."
This blog post mainly refuted the view that "neural networks need massive amounts of data, therefore they are unlike the human brain."
Parascandolo argued that the fact humans can learn to recognize dogs after seeing just a few does not mean there was no prior accumulation of experience. Long evolution has already condensed the experiences of ancestors interacting with the world into the structure and inductive biases of the human brain.
From this perspective, the large-scale pre-training of neural networks can be analogized to biological evolution, while model fine-tuning and in-context learning are closer to the learning an individual does in a lifetime. GPT-3 has read far more text than any individual, but its pre-training served to compress massive experience and shape efficient learning capabilities. Therefore, one should not directly compare the model's entire pre-training data with the small number of learning samples an individual receives after birth.
This understanding also implies that continuing to scale data and compute could still bring significant improvements. He was drawing an analogy about the roles they play, not claiming that gradient descent and biological evolution share the same specific mechanisms.
What's this guy's background?
This guy is quite low-key. His latest post on X dates back to November 2024, when he was recruiting two Research Engineers (RE) and Software Engineers (SWE) for o1.
According to his personal homepage, Parascandolo's research experience has always revolved around generalization, planning, and reasoning.
In 2017, he began his Ph.D. at the Max Planck Institute for Intelligent Systems and ETH Zurich, supervised by Bernhard Schölkopf and Thomas Hofmann. His doctoral research focused on OOD generalization in deep learning.
During his Ph.D., he did internships at Google X in Mountain View and DeepMind in London, participating in research on automated design on ultra-large-scale simulators at the former, and research on divide-and-conquer Monte Carlo tree search at the latter.
After graduating with his Ph.D. in September 2021, he directly joined OpenAI. Initially, he entered the reinforcement learning team led by John Schulman, then moved to the algorithms team where Mark Chen was, and later joined the 🍓 (strawberry) team led by Jerry Tworek. The strawberry team was the internal codename for the o1 project.
In 2023, he participated in GPT-4 development and formed a new team continuing reasoning research. Later, he participated in the foundational research for OpenAI o1 and o3, promoting the scaling of reward modeling, environment construction, and general reasoning algorithms. Part of the algorithm description is still covered by black blocks on his page.
Parascandolo didn't get the MIT professorship from that 2020 interview; he went to OpenAI instead.
Four years later, he helped build o1.
Life is always full of delightful surprises.
References:
https://x.com/giambattista92
https://sites.google.com/view/giambattista-parascandolo/home
https://gibipara92.github.io/2021/06/09/backprop-evolution-two-dogs.html
https://docs.google.com/presentation/d/1edd2GkAP31wNygaev3DO8gE1t2lgsx1z8TeVpBKqlYA/edit?slide=id.gc772610149_0_179#slide=id.gc772610149_0_179
This article is from the WeChat public account "Almost Human" (ID: almosthuman2014), author: Yang Wen
