Robot Policy Evaluation Gates

Robot teams deploying VLA policies on hands, arms, or humanoid platforms should add release tests that isolate three failure modes: tracking moving objects, remembering earlier interactions, and sensing contact under occlusion. A compact bench can include a conveyor catch, an object-in-box selection task, and insertion or deformable grasping with tactile or joint-torque logs. Pass criteria should include task success, contact-event timing, memory-dependent choice accuracy, and per-step latency on the target GPU.

RLDX-1 gives a concrete template. The policy adds multi-frame motion processing, a memory module storing past cognition-feature chunks, and tactile or torque inputs in a separate physics stream. Its reported gains are largest on tasks where a current-image policy is likely to fail, including 91.7% success on ALLEX Object-in-Box Selection and over 87.5% on conveyor-belt fast-object catching. The report also treats inference speed as a deployment metric, cutting per-step latency on an RTX 5090 from 71.2 ms to 43.7 ms.

Teams using robot video world models for planning should add a behavior-scored reward model to their training and evaluation loop. The useful check is simple: score generated rollouts for instruction following, manipulation success, action-outcome consistency, temporal consistency, contact realism, and physics adherence, then compare those scores against pixel metrics and downstream plan quality. A small distilled reward model is more practical for repeated training runs than an 8B judge.

RoboAlign-R1 shows the shape of the workflow. The authors fine-tune Qwen3-VL-8B-Thinking into a six-dimension judge, distill it into a 98M reward model, and use it for GRPO post-training. The same paper adds Sliding Window Re-encoding for long rollouts by decoding the last predicted frame, re-encoding it as fresh context, and continuing generation with a shorter active history. The reported result is higher RobotWorldBench score than iVideoGPT, better scores on all six behavior dimensions, and improved long-horizon pixel metrics with about 1% added latency.

World-model teams comparing text-controlled, keyboard-like, and camera-parameter models should build a shared action-input harness before running leaderboards. The harness should map each evaluation action into text commands, one-hot controls, and camera intrinsic or extrinsic parameters, then run the same tasks for action response, trajectory following, memory, and camera following. This reduces ambiguity when one model accepts “move forward” and another expects low-level control IDs or camera poses.

iWorld-Bench supplies a ready specification for that harness. It defines 27 translational and 27 rotational action IDs, focuses evaluation on 81 common combined actions, and builds 4,900 test tasks across action-control difficulty, memory, and camera following. The paper also reports broad coverage across four viewpoints, nine outdoor weather types, five indoor lighting types, and 18 simulator environments, which is useful for finding models that pass a narrow scene test but fail under changed viewpoint or conditions.