LeCun's world model startup, AMI (Advanced Machine Intelligence), has just quietly delivered its answer.
In the latest paper, H-JEPA, researchers from AMI, New York University, INRIA Paris, and Brown University introduced a hierarchical world model for visual planning.

H-JEPA stacks multiple JEPAs (Joint Embedding Predictive Architectures) layer by layer, allowing each layer to predict the state after different time spans within its own representation space.
During planning, the high-level layer first formulates a rough plan, then passes the predicted intermediate states as subgoals to the low-level layer. The low-level layer takes over and refines the plan step by step, ultimately generating executable actions.
Through this division of labor, the world model can both plan for long-term goals and handle immediate actions.
In the Visual AntMaze simulated maze task, the three-layer H-JEPA achieved a success rate of 73%, compared to 18% for the single-layer model.
Simultaneously, in several simulation tasks, the hierarchical model achieved better performance with less computational cost for planning.

Besides delivering tangible test results, H-JEPA also continues the open-source path that LeCun emphasized from the startup's inception.
The team has simultaneously open-sourced the code and pre-trained model weights, allowing researchers to directly load the model and replicate planning experiments.

Why Does a World Model Need Hierarchical Planning?
LeCun has actually been talking about this for a long time.
Back in 2022, while still serving as Chief AI Scientist at Meta, he proposed the concept of hierarchical JEPA:
Let the model predict at different levels of abstraction and time scales, and then implement complex goals as actions through hierarchical planning.

This might sound a bit abstract at first, but it's easy to understand using an example LeCun often cites.
Suppose you are sitting in your office at New York University, planning to go to Paris. When planning the entire trip, you focus on the destination, flights, and schedule.
After deciding on the itinerary, you start planning further: how to leave the office, go downstairs, and take a taxi to the airport.
When you actually walk out of the office, the information you focus on becomes the ground in front of you, the stairs, and where to place your next step, which foot to move first.

△
These planning levels cover different time spans and require different information. When planning the flight, the height of a specific step on a staircase is not important; when walking down the stairs, this detail directly determines how you should step.
This is not a characteristic unique to human planning. For robots, the high level needs to understand task objectives, environmental relationships, and future directions, while the low level needs to master specific actions and body control details.
For example, a quadruped robot may have already reached the target location, but if its leg posture differs from the target image, the model might still consider itself far from the goal due to these differences.

So, the question becomes:
How to let different levels learn information representations suitable for themselves, and then gradually implement high-level plans into low-level actions?
How Exactly Does H-JEPA Work?
Specifically, to solve the above problem, H-JEPA configures three components in each layer to handle state, action, and prediction respectively.

Among them, the state encoder is responsible for extracting the representation of the current environment. The lowest layer directly reads images, while higher layers further abstract based on lower-layer representations, retaining information more important at that time scale.
The action encoder is responsible for describing the changes brought about by actions. The low layer represents specific actions, while the high layer compresses a sequence of continuous actions into more coarse-grained changes.
The predictor combines the current state and action to predict future states. Different levels cover different time spans. In the paper's experiments, one step at a higher adjacent layer corresponds to two-step predictions at the lower layer.
Regarding model training, the layers above are learned jointly through end-to-end training, with different levels connected via subgoals.
The intermediate states predicted by the high level become subgoals for the low level. The low level transforms its predicted states into the high-level representation space, compares them with the subgoals specified by the high level, and then adjusts its actions accordingly.
However, when training the world model, there is a classic problem that needs to be addressed: representation collapse.
If the encoder compresses all inputs into the same vector, and the predictor only outputs this vector, it can achieve very low prediction error, but the model hasn't actually learned any useful information.
To address this, H-JEPA adopts the SIGReg proposed by LeCun's team in their previous LeJEPA work, constraining the representation distribution to prevent different states from being compressed into the same vector.

This year's LeWorldModel applied this method to world model training. H-JEPA builds upon it as the lowest layer, adding more JEPAs on top to form a structure trained jointly and planning from top to bottom.

In experiments, the researchers further observed information trade-offs at different levels. In the maze task, the high level gradually weakened the leg posture but still retained the robot's position information.

However, hierarchy also has its applicable conditions. The two-layer model performed well in the Push-T task, but adding a third and fourth layer led to decreased performance.
The paper suggests that shorter task trajectories might limit the training data available to higher layers.
LeCun, Still the Same LeCun
Overall, these designs of H-JEPA continue a fundamental proposition of JEPA: performing prediction in the representation space.
JEPA stands for "Joint Embedding Predictive Architecture." LLMs typically learn to generate content by predicting the next token; visual JEPA first encodes images or videos into internal representations, then predicts the representations of target parts, without needing to generate image pixels one by one.

In world models like H-JEPA, the model also incorporates actions to predict the representations of future states. The planner then compares the consequences of different actions to find a path towards the goal.
LeCun has always believed that understanding the world, predicting the consequences of actions, and planning are essential capabilities for achieving human-level intelligence. His assessment of LLMs has always revolved around this point.
In September this year, at the EC CV conference in Malmö, Sweden, his presentation slides once again featured that familiar piece of advice:
If you are interested in human-level AI, don't work on LLMs.

After this slide spread on social media, some doubted if it was fake. LeCun replied:
Guaranteed real.

He later added that LLM-based AI tools are very useful and everyone is using them. But he believes that LLMs alone cannot lead to human-level intelligence.
Now, this research philosophy has also become AMI's entrepreneurial direction.
In March this year, AMI announced the completion of a $1.03 billion funding round, with a pre-money valuation of $3.5 billion.
The company is chaired by LeCun, with Alexandre LeBrun as CEO, Saining Xie as Chief Scientist, Pascale Fung responsible for Research and Innovation, and Michael Rabbat responsible for World Models. Rabbat is also one of the authors of this H-JEPA paper.
However, it must be said that while research results are research results, another expectation for AMI from the outside world remains its products.
At least from the current official website, the company primarily showcases its technical direction, team, and recruitment information, with no public-facing product portal visible yet.

On the other hand, World Labs, which also bets on world models, has reached a new turning point.
On September 28th, AMD announced an all-stock acquisition agreement with World Labs, founded by Fei-Fei Li, worth approximately $8.2 billion.
It can be said that one side is moving towards integration with chip companies, while the other continues to advance along its own research path.
But with Fei-Fei Li's world model startup achieving a milestone, is Yann LeCun's major result also just a matter of time...?
References
[1]https://venturebeat.com/technology/h-jepa-teaches-world-models-to-plan-at-multiple-levels-of-abstraction
[2]https://arxiv.org/pdf/2610.06805
This article is from the WeChat public account "QbitAI", author: henry





