Robot Rollout Decision Checks

Robot teams that already run smoke tests before deployment can keep those rollouts as selection data for each task and perturbation. RouterVLA shows a simple probe-success rule selecting among frozen experts reached 0.6149 held-out success on LIBERO-Plus, compared with 0.4686 for the global best expert. The result is useful because learned scorers added little over the simple rule, while same-trial reuse inflated measured gains.

A practical version would store each candidate checkpoint’s probe outcomes, rollout length, duration, termination behavior, and missing-statistic flags, then choose a policy for the target condition with a trial-disjoint validation rule. The cheap check is to replay the lab’s existing commissioning logs through this rule and compare against the single-checkpoint choice on held-out rollouts.

Latency-tolerant manipulation tasks can add an execution wrapper that samples several reasoning-action or action-segment candidates, scores them before execution, and checks whether the observed state matches the predicted state after execution. E-TTS gives one version with reasoning-action joint sampling, vision-language verifiers, a history buffer, and feedback when a batch is rejected. PhysReflect-VLA gives a physical version with forward and inverse models that score candidate transitions and a reflector that issues corrective guidance after state mismatch.

This is a concrete adoption path for teams with a trained VLA policy that fails through bad contact, geometry violations, or accumulated action error. A first test can run the wrapper only on slow object rearrangement or insertion subtasks, then compare success, rejection rate, added latency, and recovery after mismatch against the base policy.

Robot evaluation should separate a clean completion from a completion that includes collisions, unsafe proximity, bad sequencing, or constraint violations. ForesightSafety-VLA scores each rollout as safe success, unsafe success, safe failure, or unsafe failure, then adds cumulative safety cost, risk exposure time, and safety-adjusted success rate. Its reported baselines show why the split matters: OpenVLA-oft had the best listed safety-adjusted success rate at 0.35, while unsafe success still appeared at 0.06.

A deployment review can add this as a gate on top of task success. Annotators or automated monitors would tag hazards during the whole trajectory, then report unsafe success as its own line item. The first useful check is small: rescore recent successful rollouts for contact, clearance, hot-zone, spill, and temporal-precondition violations, then see how many “successful” runs would fail a safety-adjusted review.