Robot learning gains come from predicting consequences and aligning control coordinates

Future scene changes are becoming direct training targets for control. Xiaomi-Robotics-U0 generates embodied observations and manipulation videos; adding its synthetic data raises out-of-distribution policy success from 36.9% to 63.2%. WALA learns semantic and geometric future deltas from action-free video, then connects those latent changes to executable actions. It reaches 75.2% average success on RoboCasa, versus 54.2% for its base policy. Lumo-2 follows the same design logic by predicting action-relevant latent dynamics before producing action chunks, though its excerpt does not provide numerical benchmark margins.

Two methods reduce the burden of learning camera-to-action geometry implicitly. Robot-centric pointmaps attach robot-frame XYZ coordinates to the image grid. They raise π₀.₅ success on 24 RoboCasa tasks from 55.3% to 62.9%, with larger gains under unseen camera placement. Pix2Act instead predicts continuous gripper-keypoint paths in two image planes and triangulates them into 3D actions. It achieves 75.2% mean success across ten MimicGen tasks, 12.1 points above the strongest reported baseline. Both results support explicit spatial grounding as a practical route to viewpoint tolerance.

Human demonstrations are supplying useful structure before expensive robot data enters the loop. WALA extracts action-relevant scene evolution from videos that contain no motor labels; with only 10% of labeled demonstrations, it reaches 53.9% success versus 18.1% for its base policy. Regrind starts with one human demonstration, preserves hand-object contact relationships during retargeting, and uses residual reinforcement learning for physical refinement. Across two robot hands and two tool-use tasks, simulation success ranges from 98.7% to 99.8%, with zero-shot hardware transfer claimed but not numerically reported in the inspected excerpt.