Deployable robot policies need online learning, explicit plans, and fast perception

LWD shows the strongest real-world deployment result in the set. It starts with a pretrained VLA policy, sends checkpoints to a shared fleet, collects autonomous rollouts plus optional human interventions, and retrains on mixed offline and online replay. The reported evaluation uses 16 dual-arm robots across 8 manipulation tasks, including 3–5 minute tasks such as Gongfu tea, cocktails, and fruit juice. A single generalist policy reaches 95% average success after a few hours of online interaction.

The key research signal is the use of failed trials and partial progress as training data. Distributional Implicit Value Learning fits a value distribution over replay actions, then uses an adaptive high-quantile target for temporal-difference learning. That design is meant to propagate sparse rewards across long tasks without making every rollout a clean demonstration.

IVLR makes the robot’s task plan visible before execution. Each trace stage pairs a text subgoal with an RGB keyframe, generated once from the initial observation and instruction. During control, the cached trace is combined with live camera observations to predict actions.

The ablation results make the trace useful beyond presentation. On LIBERO-Long, no trace reaches 37.7% success, text-only traces reach 62.0%, vision-only traces reach 68.4%, and full interleaved traces reach 92.4%. The method also reports 95.5% average success on LIBERO and 59.4% on SimplerEnv-WidowX. The cost is upfront planning time: full trace generation takes about 10 seconds on one NVIDIA H20 GPU, then cached execution runs at 10 Hz.

Two papers from the same author group put spatial tracking into compact real-time policies. MSACT adds self-supervised 2D attention-point tracking to ACT for bimanual fine manipulation. On 400 real-world ALOHA trials across 4 tasks, it reports 53.00% success, compared with 23.25% for ACT, while keeping latency at 45.40 ms, almost identical to ACT’s 45.34 ms.

The stereo mobile-manipulation variant uses left and right RGB images, shared attention extraction, and a hierarchical LSTM action predictor. Across 4 real-world tasks with 50 randomized trials per task, it reports 85.0% average success, compared with 46.0% for ACT and 28.5% for Diffusion Policy. Disturbance tests also favor the structured attention design: 76.8% overall success across 560 trials, compared with 24.8% for ACT.

VLADriver-RAG applies retrieval to autonomous driving by indexing semantic traffic graphs rather than raw images. The planner retrieves similar traffic topology and interaction history, then predicts path and speed waypoints. On Bench2Drive, it reports an 89.12 Driving Score and 70.42% Success Rate, above ORION’s 77.74 and 54.62% on the same benchmark.

The interpretability paper adds a diagnostic layer for VLA policies. Interventional Significance Score masks visual regions and measures action change; Nuisance Mass Ratio measures how much top saliency falls on task-irrelevant areas. Across 41 AGNOSTOS tasks, NMR@10 has a Pearson correlation of -0.77 with task success, so high nuisance attribution tracks worse unseen-task performance.

The Hamiltonian world-model paper is more conceptual. It argues for latent phase states, energy-like dynamics, and action-conditioned rollouts for planning. The corpus summary reports no benchmark numbers, so its value here is a design hypothesis rather than empirical evidence.