Execution supervision for robot policies

Robot teams working on long household or warehouse sequences can now build a supervisor layer that replans after every step from the current camera view, keeps a short done-and-remaining memory, and sends the action policy a simple 2D trace for the next local move. LoHo-Manip gives a concrete recipe for this split between task management and execution. The value is operational: when a grasp misses or an object shifts, the failed subtask stays in the remaining plan and the trace updates on the next cycle, so recovery stays inside the normal control loop.

A practical first version does not need a new foundation model. Take an existing short-horizon VLA or diffusion policy, add a planner that rewrites the remaining subtasks after each observation, and render a trace overlay into the executor input. The cheap check is a replay set of multi-step tasks with injected small failures such as off-center grasps or moved objects. Measure whether the system resumes the intended sequence without a manual reset. This is credible now because LoHo-Manip reports gains on long-horizon reasoning and trajectory prediction benchmarks, including 63.1 on RoboVQA and 56.7 on EgoPlan2, with a receding-horizon planner explicitly designed to preserve failed steps in memory until they are completed.

Post-training generalist robot policies can move into a simulator-first correction workflow where operators intervene near failure states inside a learned world model, branch several fixes from the same cached state, and only spend robot time on validation runs. Hi-WM gives enough detail to build this as a data collection product for labs already fine-tuning Pi0, Diffusion Policy, or similar controllers.

The user pressure is cost. Real-world correction loops require resets, scene setup, and operator supervision for every bad rollout. Hi-WM shows a world model can carry part of that loop if it is aligned closely enough to the action space and edge cases. On three real robot tasks, the paper reports average success gains of 37.9 points over the base policy and 19.0 over a world-model closed-loop baseline, with Pearson r = 0.953 between world-model evaluation and real performance. That supports a narrow deployment path: use the simulator to decide which failure states deserve human correction data, then retrain and spot-check on hardware. A cheap validation is one task with frequent late-stage failures, such as deformable object routing, comparing operator minutes per successful correction collected in simulation versus on the robot.

Robotics eval stacks for household manipulation need an execution-safety review path that records collisions, dropped support objects, unstable placements, and grasp failures alongside task completion. The immediate build is a scorer and video audit tool for BEHAVIOR1K-style runs, using penalties like sQ and seQ plus a small taxonomy for reviewer labels.

This is useful for teams selecting checkpoints and reporting progress. Final-state scores can hide the run quality that matters for deployment and for debugging. The BEHAVIOR1K audit reviewed 500 recordings and found grasp failure was the most common error, with collisions also frequent. When safety penalties are applied, average RLC score drops from Q = 0.256 to sQ = 0.239, and Comet drops from 0.192 to 0.173. A simple product test is to rescore a recent batch of benchmark videos and rank checkpoints by both completion and safety-adjusted score. If the ordering changes, the review path is already paying for itself.