Robot policy execution and robustness

A streaming control wrapper for existing VLA policies looks buildable now. The operational pain is stop-and-go execution on real robots when observation, action generation, and execution wait on each other. StreamingVLA gives a concrete template: overlap those stages, switch chunk prediction to action flow matching so the robot can execute the next small action immediately, and gate early observation with a saliency predictor. On LIBERO with pi_0.5 as the base model, the paper reports the same 97.1% average success as the baseline while cutting time per action from 74.5 ms to 33.7 ms and reducing the halting gap from 232.3 ms to 76.1 ms. The more aggressive AFM+AEO variant reaches 31.6 ms and 36.0 ms halting gap, with success dropping to 94.9%, which gives a practical latency-quality tradeoff to tune.

The near-term product is not a new foundation model. It is a runtime layer for teams already running chunked VLA policies on edge hardware or mobile manipulators. A cheap validation check is to instrument one deployed policy with per-step timestamps, measure halt time between action bursts, then add asynchronous observation and single-step action generation on a narrow task slice such as pick-and-place. If the robot stops less often without a rise in task failure, the wrapper earns its keep before any retraining campaign.

A latent-intent adapter between a VLM and a low-level controller is becoming a practical build for manipulation teams that cannot afford large robot demonstration sets. DIAL uses a world-model objective to predict future visual features at horizon H=16, then feeds that latent future into a policy that generates a 16-step action chunk. The key point for adoption is not only architecture elegance. The paper reports state-of-the-art results on RoboCasa GR1 Tabletop with 10× fewer robot demonstrations, using 2,400 trajectories against a 24,000-trajectory full-data regime, and also reports zero-shot transfer gains from mixing robot data with 27,419 EgoDex human trajectories.

This suggests a concrete workflow change: treat future-state prediction as the supervision interface for high-level intent, then fine-tune the controller jointly so action gradients still shape the planner. Teams working on table-top manipulation or humanoid pick-and-place can test this without a full platform rewrite. Train the controller first against ground-truth future features, then swap in predicted latent futures and compare success under a fixed demonstration budget. If the lower-data run holds up on unseen objects or object combinations, the adapter has immediate value for data-constrained robot training.

Instruction robustness testing needs to move into the default evaluation loop for VLA systems that are close to deployment. LIBERO-Para shows that current models often look solid when train and test wording match, then break on meaning-preserving rewrites. Across seven VLA configurations, success falls by 22.8 to 51.9 percentage points under paraphrasing. VLA-Adapter drops from 98.2% on LIBERO-Goal to 46.3% on LIBERO-Para. The paper also finds that 80% to 96% of failures come from planning-level trajectory divergence, which means the problem starts before contact control.

The practical build is a paraphrase regression harness tied to instruction templates, synonym swaps, and indirect command forms, scored with both raw success and PRIDE. Teams can run it after fine-tuning and before real-world demos. The first users are evaluation and safety leads who need to know whether a policy has memorized wording. A cheap check is to take one high-performing policy, generate paraphrases only for object references in a small task set, and compare trajectory class or first waypoint choice before full task success. If object-name substitutions change the plan early, language robustness is an immediate blocker for field use.