Closed-loop neural rollout regression tests for VLA policy releases
Robot teams can add a pre-release regression stage that runs each candidate VLA policy inside an action-conditioned video world model, scores rollouts with a 0–5 task-progress VLM rubric, and sends only disputed or high-risk cases to real hardware. RoboWorld reports 4,186 generated rollouts across eight open robot policies and a close match to the RoboArena real-world ranking, with Pearson r=0.989 and Spearman rho=0.970. Its task-progress scoring also beat binary success scoring in the reported ranking comparison.
A practical first version can start with the team’s own held-out tasks: replay policy actions through a learned video model, score progress per rollout, and compare the resulting rank order with a small weekly batch of real robot trials. The system should log world-model artifacts separately, since RoboWorld’s own design uses wrist-view checks to detect generated-video failures. The value is release triage: catching policy regressions across objects, camera views, and task variants before using scarce robot time.