Embodied policy reliability checks

Robot teams fine-tuning OpenVLA-like policies can add an action-distribution regularizer before they spend more on data collection. The usable change is narrow: keep probability mass over a small neighborhood of good actions during finetuning, then measure whether that improves held-out task success and OOD variants in the target environment. The FAN paper gives a concrete target for that check. On ManiSkill supervised finetuning with OpenVLA, in-distribution success rises from 78.1 to 89.8, and average OOD success rises from 58.1 to 63.3. That is large enough to justify a lightweight ablation in any manipulation stack that still trains against a single exact action label.

This also points to a practical support layer for deployment teams: inspection of action sharpness during finetuning. If the policy collapses to narrow peaks, that is a candidate failure mode for small demonstration sets and minor execution shifts. A cheap first test is to replay the current finetuning set, compare log-likelihood-only training against the same setup with a local Gaussian prior around the preferred action, and track success on small visual, semantic, and execution shifts. The controller-gain paper strengthens the case for this workflow because it shows that learnability depends on the control interface as well as the policy objective. In behavior cloning, the best closed-loop success appears in compliant, overdamped gain settings, and torque-to-position retargeting preserves at least 90% success with joint-position MSE below 1e-3 across gain settings up to 25x decimation. Teams adapting pretrained VLAs to a new arm or controller can treat action-tolerance tuning and gain selection as one finetuning problem, not two separate cleanup steps.

Driving teams building world-action models can justify a geometry-first planning stack now that there is a concrete closed-loop result behind it. The build is specific: predict depth before future video and action, use that depth map as an explicit scaffold for imagination and planning, and keep the generators modular so planning-only and world-generation modes can share the same backbone. DriveDreamer-Policy gives a clean benchmark anchor for this design. On Navsim v1 it reaches 89.2 PDMS, ahead of PWM at 88.1, WoTE at 88.3, DriveVLA-W0 at 88.4, and AutoVLA at 89.1. On Navsim v2 it reaches 88.7 EPDMS, with the paper stating a 2.6-point gain over the previous method shown in the table.

The practical workflow change is in evaluation as much as model design. A visually convincing rollout is not enough if the representation misses free space, layout, or occlusion structure. A simple adoption path is to run a depth-first ablation against an existing future-video planner, then compare closed-loop planning scores and a small set of occlusion-heavy cases. The same logic appears in WAV for robot world models: verification improves when future-state plausibility and action reachability are checked separately, and the paper reports 2x sample efficiency across nine tasks plus an 18% downstream policy gain. Together, these results support a planning workflow where geometry and reachability get explicit intermediate checks, not just a final action loss.

Anyone evaluating VLA manipulation systems on physical tasks needs an object-surface attack test in the release checklist. Tex3D is concrete enough to move this from a research warning to a standard red-team step. The attack is attached to the manipulated object's 3D texture, optimized through a differentiable rendering path, and kept effective across long episodes with trajectory-aware weighting. In simulation, the failure jumps are large across several common models: OpenVLA rises from 24.1% to 88.1% under untargeted attack, OpenVLA-OFT from 4.7% to 76.0%, and pi0 from 4.6% to 71.8%. On OpenVLA spatial tasks, failure reaches 96.7% under targeted attack.

The workflow change is straightforward. Before shipping a new checkpoint, take a small set of benchmark objects, optimize surface textures against the frozen policy in simulation, and record failure rates by task family and object category. That gives teams a way to find policies that rely on brittle visual shortcuts even when standard perturbation tests look clean. The paper does not yet give a defense recipe with verified recovery numbers, so the near-term product is an evaluation harness and acceptance threshold, not a robustness claim. For teams selling VLA systems into warehouses, labs, or homes, that harness is easier to adopt than waiting for a full training-time defense stack.