Geometry Checks for Robot Training

Manipulation teams should add a contact-region evaluation slice to VLA policy testing. The useful test cases are tasks where a small 3D error changes the outcome: hanging a mug by its handle, handover, pressing a tool, placing an object through an opening, and moving near obstacles. The evaluation should log success, collisions, missed grasps, contact-point error, end-effector clearance, and whether the model attends to the object region that controls the action.

STARRY gives a concrete design to copy and test. It predicts future depth and end-effector positions, unprojects predicted depth into 3D, computes token distances to the predicted end effector, and uses those distances to bias action-to-video attention. The paper reports 93.82% clean and 93.30% randomized average success on 50 RoboTwin 2.0 bimanual tasks, plus 70.8% average success in real ARX R5 bimanual experiments using 50 demonstrations per task and 20 evaluation rollouts per method. The biggest value is the failure analysis it enables: when a policy misses a handle or collides near an opening, the team can inspect the predicted depth, end-effector path, and attention weights before collecting more demonstrations.

Teams adapting video diffusion models for robot control should run a latency test that decodes actions earlier than video. The test is simple: train or fine-tune a model that predicts future RGB-D frames, robot state, and a block of future actions, then compare control latency and task success under different denoising budgets for actions and video. The pass condition is practical, not visual: the robot must produce actions fast enough for closed-loop control while depth and video predictions remain good enough for debugging and planning.

X-WAM is a useful reference implementation. It fine-tunes Wan2.2-TI2V-5B on multi-view robot data, predicts 8 future RGB frames, 8 future states, and 32 future actions, and adds a depth branch without doubling the token sequence. Its Asynchronous Noise Sampling trains video and action noise levels from a coupled distribution so inference can use fewer denoising steps for actions. Reported results include 79.2% average success on RoboCasa across 24 tasks and 90.7% on RoboTwin 2.0 Randomized. A lab can reproduce the most relevant part without matching the full scale: measure milliseconds per action, success rate, and geometric prediction quality as the action denoising budget is reduced.

Simulation teams using generated 3D assets should add an import gate before assets enter robot-training runs. The gate should reject objects that have only shape and texture. A useful generated asset needs valid mesh geometry, collision geometry, joint definitions, joint limits, mass, friction, material behavior where relevant, and export to formats such as URDF, MJCF, or USD. The check should load the asset in the target simulator, execute its joints, drop or push it under physics, and verify that the file can be reused across the team’s training jobs.

The 3D generation survey names the adoption blocker clearly: visual quality does not guarantee physical validity. It defines simulation readiness around geometry, physical parameterization, kinematic executability, and simulator compatibility, and connects generated content to engines and formats including MuJoCo, Isaac Sim, Habitat, AI2-THOR, OmniGibson, PyBullet, ManiSkill3, Genesis, URDF, MJCF, and USD. That turns asset generation into a testable workflow issue for robotics groups. A small validator script and simulator smoke test will catch many failures before policy training consumes bad scenes.