Li Feifei's Latest Article: When Video Generation, Robotics, and NVIDIA All Claim to Have 'World Models,' We Need a Taxonomy
"World Model" has become a widely used yet ambiguous term in AI. Drawing from the classic POMDP framework (agent → action → state → observation), this article proposes a functional taxonomy to clarify the concept. It identifies three distinct types, categorized by their output in the perception-action loop:
1. **Renderers**: Output visual observations (pixels). These models, like advanced video generators, prioritize visual fidelity but often lack underlying physical accuracy.
2. **Simulators**: Output the state of the world (geometry, physics, dynamics). They provide a structurally accurate representation for professionals (e.g., architects) and serve as training environments for robots and AI agents.
3. **Planners**: Output actions. Given an observation and a goal, they determine what an agent should do next, closing the perception-action loop (e.g., vision-language-action models).
While renderers are currently the most commercially mature and planners are the most aspirational, the article argues that **simulators are the crucial, underappreciated hub**. By working at the level of geometry and physics, a simulator can project upwards to create visuals for humans and downwards to predict action consequences for agents.
The future lies in the convergence of these three functions. Emerging research and products, like World Labs' Marble model which outputs both visual splats and physical collision meshes, are beginning to blur these boundaries. The logical endpoint is a unified world foundation model capable of rendering, simulating, and planning based on a shared understanding of spatial and temporal structures—ultimately enabling machines to understand, imagine, and interact with the physical world.
链捕手07/05 09:12
