Over the past two years, the competitive focus in the humanoid robot sector has been shifting from the hardware of the entire machine further to model capabilities.
Manufacturers have been releasing new products intensively, with videos of backflips, dancing, and marathon running frequently dominating social feeds. Behind the fanfare, however, the industry has gradually reached a consensus: what determines the upper limit of a humanoid robot's ability is no longer just its joints and motors. The capacity to understand the environment, predict changes, and coordinate the entire body to complete tasks is becoming the key to achieving generality.
World models, VLA, and humanoid robot foundation models have thus become some of the most important technical directions in this field over the past two years.
Amidst this excitement, three major challenges have remained.
First, the cost of collecting real-world demonstration data for humanoid robots is high. Collection requires synchronous recording of first-person video, proprioceptive data, and executable whole-body commands. Constrained by teleoperation difficulty, safety risks, hardware availability, and environmental diversity, it is difficult to accumulate large-scale, high-quality data in a short time.
Second, many existing world-action models follow the pixel-level video prediction approach, which is computationally expensive, with significant capacity consumed by image details weakly correlated with control. The robot's own rapid movement and viewpoint jitter further amplify visual prediction noise.
Third, many existing solutions still model upper-limb manipulation and lower-body locomotion separately, resulting in insufficient coordination between the upper and lower body and difficulty in supporting natural, fluid whole-body control.
Against this backdrop, Embodied Intelligence company Zhi Zai Wu Jie has released Being-M0.7. This is the world's first Latent World-Action Model (Latent WAM) for whole-body mobile manipulation of humanoid robots, and it is also the industry's first time extending the capabilities of latent world models from desktop dexterous manipulation to whole-body mobile manipulation.
- Paper Link: https://research.beingbeyond.com/being-m07/being-m07.pdf
- Project Page: https://research.beingbeyond.com/being-m07
It was pre-trained on over 10,000 hours of human-centric mixed-modal data, then adapted to a specific embodiment using a small amount of real-world demonstration data, and has successfully completed multiple challenging whole-body mobile manipulation tasks on a physical humanoid robot.
From Being-H to Being-M: A Continually Materializing Technical Path
Behind Being-M0.7 lies a technical path that Zhi Zai Wu Jie has steadfastly pursued for years.
This company is one of the earliest embodied intelligence firms globally to bet on the large-scale human video training path, concurrently developing two main model lines: general dexterous manipulation and general mobile dexterous manipulation. It is also the first team in China to launch a natively embodied implicit world-action model.
The core judgment of this path is that real-world robot demonstration data is expensive and scarce, making it difficult to scale continuously like internet text and video. In contrast, humans interact with the physical world from a first-person perspective daily. This vast amount of human behavioral data contains rich priors about scene evolution, object dynamics, and body coordination. Rather than waiting for robot data to slowly accumulate, it is better to first let models learn how the world works from human experience, and then transfer this knowledge to specific robot embodiments.
Being-H0.7, released in April this year, validated the feasibility of this judgment for dexterous manipulation. The model scaled training data to 200,000 hours of human video, achieved an overall first-place ranking globally in 6 international benchmarks, topping 4 of them, and became the first general embodied world model covering seven key dimensions: cross-embodiment, cross-scenario, continuous dynamics, fluids, deformable objects, physical laws, and contextual reasoning.
Being-M0.7 is the latest result of this implicit world-action model path.
If the Being-H series answers how hands manipulate the world, Being-M0.7 answers how the entire body coordinates locomotion and task execution in the world. Humanoid robot mobile manipulation (loco-manipulation) requires the model to simultaneously decide where to go, how to orient the body, how to coordinate arms and legs, and how to maintain posture stability. This is a problem highly coupled in both the temporal and body dimensions, and a critical hurdle that general-purpose humanoid robots must overcome.
Being-M0.7 is an implicit world-action model. It is first pre-trained on first-person video and human motion data using a Mixture of Transformers (MoT) architecture; then, through action expert post-training, it is adapted for control on robot trajectory data from diverse whole-body manipulation tasks.
Unlike many world models relying on pixel-level video generation, Being-M0.7 predicts future environmental states in a latent space and couples them with a compact whole-body motion representation. Pixel-level prediction is computationally expensive, with much computational power wasted on appearance details weakly related to control. Severe self-motion and viewpoint jitter in the first-person perspective further fill predictions with noise. Latent space prediction focuses modeling capacity on semantic states, object layout, and scene evolution—structural elements truly relevant to control—retaining the essence of a world model anticipating the future while significantly reducing computational overhead.
How Does Physical Understanding Translate into Whole-Body Action?
Whether the model truly possesses whole-body mobile manipulation capabilities ultimately must be tested in real-world scenarios.
Zhi Zai Wu Jie released four real-world demos centered around Being-M0.7, covering four highly challenging scenarios: liquid interaction, mirror reasoning, long-horizon tasks, and obstacle avoidance with occlusion.
These tasks collectively examine one question: Can the robot, based on predictions of the environment and its future changes, continuously generate whole-body actions appropriate to the current scene?.
Fishing in a Fish Tank
The robot walks to an aquarium and uses a handheld net to scoop a toy fish from the water. Liquids have no fixed shape, flow, exert buoyancy and drag on immersed objects, and water surface refraction causes the visual position of underwater targets to shift. The robot must understand the interaction between water, net, and fish, coordinating its arm to complete a tool-based capture of a dynamic target while visual information is distorted by the water. This task precisely tests future state prediction, tool use, and motion coordination under uncertain object dynamics.
For the fish-scooping segment of the task, Being-M0.7 succeeded in 3 out of 5 tests. In comparison,
was 2/5, while GR00T-N1.6 was 1/5.
Mirror Object Retrieval
A box sits in front of the robot, open only on the back and sides. The item inside is completely invisible from the robot's own perspective; the only clue is the reflection in a mirror ahead. The robot needs to infer the hidden object's position in real 3D space based on the mirrored view, then approach the box and reach in to grab it. This requires the model to understand the spatial relationships and principles of mirror reflection among the mirror, box, and object, converting indirect visual evidence into executable actions under partially observable conditions.
In real-world comparative tests with
and GR00T-N1.6, under two distance settings of 0.5 meters and 1 meter, Being-M0.7 succeeded in 3 out of 5 and 1 out of 5 tests respectively, overall 4/10;
and GR00T both achieved an overall score of 1/10.
This result indicates that Being-M0.7 demonstrates stronger adaptability in partially observable tasks requiring indirect visual reasoning, whole-body approach, and fine grasping.
Mobile Object Transfer and Retrieval
The robot walks to a table, transfers a baguette from one basket to another, then picks up a bouquet from a basket and turns to leave. The task consists of multiple sub-tasks strung together. The robot needs to continuously switch between walking, grasping, transferring, turning, etc., while maintaining an ongoing understanding of the scene throughout. It tests not just single-grasp success rate, but also state maintenance in long-horizon tasks, object-level spatial reasoning, and whole-body synergy between locomotion and dexterous manipulation.
Box Carrying with Obstacle Avoidance
The robot walks forward carrying a box. When an obstacle appears ahead, it doesn't completely stop to re-plan, but adjusts its body orientation to sidestep through the narrow space between obstacles. Carrying the object partially occludes the first-person view and also alters the robot's load and center of gravity. The model needs to combine prior environmental information with real-time feedback to judge passable areas, adjust walking direction and whole-body posture, while maintaining its own balance and the stability of the carried object. Multi-directional movement, obstacle avoidance, and load-aware manipulation are unified into one closed-loop behavior here.
These demonstrations illustrate that the robot does not execute according to a fixed, open-loop trajectory, but rather continuously generates and corrects whole-body actions by combining current observations, real-time feedback, and predictions of the future.
MoT Architecture and Unified Motion Representation: Solving the Embodiment Data Scarcity Challenge
Supporting the above capabilities is a set of key designs in data and architecture for Being-M0.7.
Humanoid robots need to perform spatial perception via first-person visual information and also output future motion and control commands. High-quality human motion data typically requires motion capture equipment, while first-person paired data where vision and motion are strictly aligned is even scarcer.
If a model can only use data containing both vision and motion, the scale of trainable data would be severely limited. The problem Being-M0.7 solves is precisely how to let paired data, pure video data, and pure motion data all participate in training.
Zhi Zai Wu Jie's choice is the Vision-Motion MoT (Mixture-of-Transformers) architecture. Vision-Motion MoT retains separate modality-specific projection and processing modules for vision and motion, while enabling cross-modal interaction through shared multimodal attention. Visual state changes and continuous motion have different data distributions and need not be forced into identical parameter systems; when both modalities are present, they can exchange information within a shared context.
This allows the model to simultaneously handle three types of data.
For video-motion paired data, the model jointly learns future environmental states and motion trajectories; for pure video data, only the visual branch's training objective is computed; for pure motion data, only the motion branch is trained. Data from different sources jointly constrain the model through the same training objective, without needing to train multiple isolated single-modal systems separately.
From a probabilistic modeling perspective, paired data characterizes the joint relationship between vision and motion, while single-modal data provides marginal constraints for this joint distribution. Even data with incomplete modalities can be incorporated into the same training framework.
Overview of the Being-M0.7 training framework. Top Left: Pre-training data consists of video-motion paired data, pure video data, and pure motion data. Bottom Left: The research team constructed a unified motion representation shared between humans and humanoid robots, providing richer supervision signals and feedback for training and inference. Right: The overall model architecture of Being-M0.7.
Based on this architecture, the team built a mixed-modal pre-training dataset exceeding 10,000 hours, covering human first-person videos, first-person video-motion paired data, and pure human motion sequences.
Being-M0.7's Data Recipe. The pre-training corpus is sourced from multiple external public datasets, including Ego4D, Xperience, Nymeria, Bones-SEED, SnapMoGen, HumanML3D, and Lafan1; it also includes internal datasets.
Another key design is the unified motion representation shared between humans and humanoid robots.
Being-M0.7 proposes a unified action representation that transforms human motion data from different sources into a unified representation with the head as the root node, naturally aligning it with first-person vision. Through standardization processes like unifying coordinate systems and removing initial orientation, it reduces distribution differences between datasets and improves cross-source consistency.
Furthermore, Being-M0.7 employs a compact motion representation that retains only the head, both hands, and both feet. While preserving key interaction and contact information, it effectively bridges the morphological gap between humans and robots. This representation not only provides richer supervision signals than action labels for robot post-training but also offers motion-level feedback during inference, supporting whole-body coordinated control.
During pre-training, the model maps images to latent space via a visual encoder and uses the compact unified motion representation. The model is trained with a Flow Matching objective to jointly predict future state changes and motion trajectories based on a short segment of visual-motion history and task instructions.
For real-world robot data collection, the team set up a whole-body teleoperation system based on PICO VR. An operator wears a PICO headset, two ankle trackers, and two handheld controllers. The VR system estimates the human pose in real-time, which is then converted into executable 29-degree-of-freedom whole-body control commands for the Unitree G1 via a whole-body motion controller. While the robot executes actions via teleoperation, it simultaneously records onboard RGB camera first-person view images, proprioception, and motion control commands. This serves as the post-training fine-tuning data for Being-M0.7 on specific tasks.
Being-M0.7 real-world robot data collection system. The operator provides whole-body motion commands via VR devices; the system converts human poses into robot control commands and synchronously collects first-person images, proprioception, and motion trajectories.
Since the model has already established visual-motion priors during pre-training, the real-world robot data no longer needs to teach all motion patterns from scratch; it primarily serves two purposes: first, to ground the pre-trained priors into the specific control space of the humanoid robot; second, to learn the low-level control commands and feedback mechanisms required by the real robot. This process is handled by a lightweight Action Expert. The Action Expert reads the intermediate hidden states of the Latent WAM as high-level planning context, then combines them with current visual observations, proprioceptive information, and execution progress to generate action chunks directly executable by the robot.
During inference, the model generates future video-motion plans at a low frequency and converts their intermediate hidden states into reusable policy caches (KV Cache). The unified motion representation not only fuses visual and proprioceptive feedback but also uses the robot's latest motion state to correct the predicted whole-body motion plan, enabling the policy to promptly respond to deviations in body and end-effector motion. The Action Expert then reuses the current KV Cache to continuously generate actions at a high frequency, seamlessly incorporating the latest robot feedback when the cache refreshes. This design decouples low-frequency world planning from high-frequency action control, ensuring real-time performance while keeping the robot consistently guided by both long-term planning and real-time feedback.
A Scalable Fusion Paradigm, Towards More General Embodied Intelligence
The significance of the Vision-Motion MoT architecture extends beyond solving the training problem for Being-M0.7 alone; it establishes a sustainable, extensible multimodal fusion paradigm.
The most direct change from this paradigm occurs at the data level.
Over 10,000 hours of mixed-modal data expand the sources of supervision signals available for training humanoid robot models from expensive, scarce real-world robot demonstrations to vast human behavioral data. Loosening the data bottleneck is a prerequisite for any Scaling Law to hold.
Simultaneously, Being-M0.7 also adjusts the sequence of model learning.
Before being adapted into robot-executable commands, the model first learns visual context, future dynamics, and humanoid kinematic structure from large-scale human-centric data. Subsequently, the Action Expert translates these predictions and motion priors into control commands for the specific robot. In other words, the model first builds the capability to predict future states and body motion, then learns how to act on a specific embodiment. This constitutes a key difference between it and traditional imitation learning schemes that learn action mappings directly from robot demonstrations. The latter often start from "see something, output some action," while Being-M0.7 inserts a layer of joint modeling of future state-motion before action generation.
Moreover, this architecture does not require all newly added data to have complete vision-motion pairing. After cleaning and processing, standalone human video and motion sequences can all be incorporated into the same model. As data scale expands further, this fusion paradigm has the potential to continuously extend the capability frontier.
Placed within the industry's coordinate system, the release of Being-M0.7 perhaps signifies a shift in the competitive logic of humanoid robots.
In recent years, industry attention has focused more on whose robot body is more agile and whose motion demos are more spectacular. As hardware performance continues to improve, whether models can understand scenes, predict changes, and generate coordinated whole-body actions, and whether there exists a scalable data system behind them, will increasingly become the key differentiator.
The development of large language models has already proven that scalable data and a training flywheel often determine how far a technical path can ultimately go. Embodied intelligence stands at a similar critical point: real-world robot data cannot grow rapidly like internet corpora; where else can robots obtain the experience needed for continuous evolution?
From Being-H to Being-M, Zhi Zai Wu Jie's judgment is to let robots first learn about the world from human behavior, then translate this knowledge into actions in the real physical world.
When understanding becomes the prerequisite for action, general-purpose humanoid robots truly step out of the lab narrative and begin their journey towards the physical world of myriad industries.
This article comes from the WeChat public account "Machine Heart" (ID: almosthuman2014), author: Yang Wen
