--- kind: ideas granularity: day period_start: '2026-04-27T00:00:00' period_end: '2026-04-28T00:00:00' run_id: 4dadeab4-0fe1-4af6-908f-2a3b1e2cf429 status: succeeded topics: - vision-language-action - robot manipulation - coarse-to-fine control - robot deployment - human demonstrations - edge safety tags: - recoleta/ideas - topic/vision-language-action - topic/robot-manipulation - topic/coarse-to-fine-control - topic/robot-deployment - topic/human-demonstrations - topic/edge-safety language_code: en pass_output_id: 117 pass_kind: trend_ideas upstream_pass_output_id: 116 upstream_pass_kind: trend_synthesis --- # Robot VLA deployment controls ## Summary Robot VLA adoption is becoming a control-loop engineering problem. The useful next work is to measure action latency on target hardware, add edge correction for delayed cloud waypoints, and reuse human manipulation videos through intent-level supervision before robot-action training. ## Action-generation latency benchmark for VLA manipulation policies Robot teams testing VLA manipulation policies should add a release gate that measures action sampling latency, NFE, end-to-end observe-infer-act time, and task success on the same manipulation suite. The gate should run on the intended robot computer, then repeat after compilation, caching, sampler changes, or action-head changes. CF-VLA gives a concrete baseline for this test. It reports 96.5 average LIBERO success at NFE=2, a 75.4% reduction in action sampling latency, and 83.0 average success in real-robot experiments. Libra-VLA points to a related implementation pattern: run a heavier semantic planner less often, store coarse intent in a FIFO buffer, and keep the action refiner at the control rate. The hardware profiling paper shows why this belongs in the deployment workflow. For pi0, measured latency ranges from 102.3 ms on RTX 4090 to 246.0 ms on Jetson Thor and 920.6 ms on AGX Orin before compilation, while energy and cost move differently across devices. A cheap validation run is a two-column comparison: current sampler versus a coarse-to-fine or buffered variant, with success and latency recorded together. A sampler that saves milliseconds while lowering contact-task success should fail the gate. ### Sources - [CF-VLA: Efficient Coarse-to-Fine Action Generation for Vision-Language-Action Policies](../Inbox/2026-04-27--cf-vla-efficient-coarse-to-fine-action-generation-for-vision-language-action-policies.md): CF-VLA reports the two-step NFE=2 sampler, LIBERO success, action sampling latency reduction, and real-robot success. - [Libra-VLA: Achieving Learning Equilibrium via Asynchronous Coarse-to-Fine Dual-System](../Inbox/2026-04-27--libra-vla-achieving-learning-equilibrium-via-asynchronous-coarse-to-fine-dual-system.md): Libra-VLA describes coarse intent prediction, continuous refinement, and an inference-time FIFO intent buffer. - [Characterizing Vision-Language-Action Models across XPUs: Constraints and Acceleration for On-Robot Deployment](../Inbox/2026-04-27--characterizing-vision-language-action-models-across-xpus-constraints-and-acceleration-for-on-robot-deployment.md): The deployment study reports model-hardware latency, energy, and compilation effects for VLA inference. ## Edge waypoint realignment and LiDAR safety policy for cloud VLA navigation Teams running cloud VLA navigation should put a small edge adapter between delayed waypoints and the robot controller. The adapter needs a timestamped pose buffer, an SE(2) transform that maps stale waypoints into the current ego frame, and a local policy that chooses sub-goals using corrected look-ahead points plus LiDAR proximity. AsyncShield is a direct template. It uses five realigned look-ahead waypoints spaced 0.2 m apart and 144 LiDAR proximity values, then applies a PPO-Lagrangian policy with a LiDAR-based safety cost. Under mixed network degradation, it reports 76.7% success, 0.725 m cross-track error, and 1.3% risk exposure. Removing temporal alignment drops success to 36.7%, and removing safety constraints gives low tracking error with poor safety. The first field test can replay logged cloud-VLA packets with injected delay, jitter, and packet loss while the robot drives in a mapped obstacle course. The pass condition should combine arrival, cross-track error, and minimum-distance violations, since close tracking can still be unsafe when waypoints are stale. ### Sources - [AsyncShield: A Plug-and-Play Edge Adapter for Asynchronous Cloud-based VLA Navigation](../Inbox/2026-04-27--asyncshield-a-plug-and-play-edge-adapter-for-asynchronous-cloud-based-vla-navigation.md): AsyncShield provides the pose-buffer, SE(2) realignment, LiDAR policy, network-degradation results, and ablations. - [AsyncShield: A Plug-and-Play Edge Adapter for Asynchronous Cloud-based VLA Navigation](../Inbox/2026-04-27--asyncshield-a-plug-and-play-edge-adapter-for-asynchronous-cloud-based-vla-navigation.md): The paper abstract states the cloud-to-edge latency problem and the edge correction mechanism. - [Characterizing Vision-Language-Action Models across XPUs: Constraints and Acceleration for On-Robot Deployment](../Inbox/2026-04-27--characterizing-vision-language-action-models-across-xpus-constraints-and-acceleration-for-on-robot-deployment.md): The VLA deployment profiling paper frames latency as a closed-loop robot failure source. ## Human-video intent pretraining before robot-action fine-tuning Manipulation teams with limited robot demonstrations should test a human-video pretraining stage that predicts interaction trajectory and hand-motion intent before training the robot action head. The useful unit is transferable intent: where the hand moves, where contact happens, and how the object interaction unfolds. Robot action losses should be kept from overwriting that prior during fine-tuning. MoT-HRA gives the most concrete recipe. Its HA-2.2M dataset is built from HowTo100M, Ego4D, EPIC-KITCHENS, and Something-Something-V2, then filtered and reconstructed with hand-centric selection, MANO-style pose, depth alignment, temporal segmentation, and language labeling. The model separates an embodiment-agnostic 3D trajectory expert, a MANO-style hand-motion expert, and a robot action expert. On SimplerEnv-WidowX it reports 66.1% average success, ahead of listed baselines including ThinkACT, SpatialVLA, OpenVLA-OFT, RoboVLMs, π0-FAST, and π0. A practical pilot is to curate a small task-specific human-video subset, pretrain the trajectory and hand-motion heads, then fine-tune with the same 50 to 100 robot demonstrations used by a robot-only baseline. The comparison should include novel object placement and instruction variation, since $M^2$-VLA also reports larger gains when instructions and objects change after training. ### Sources - [Learning Human-Intention Priors from Large-Scale Human Demonstrations for Robotic Manipulation](../Inbox/2026-04-27--learning-human-intention-priors-from-large-scale-human-demonstrations-for-robotic-manipulation.md): MoT-HRA describes HA-2.2M, the three-expert split, read-only transfer, and SimplerEnv-WidowX results. - [Learning Human-Intention Priors from Large-Scale Human Demonstrations for Robotic Manipulation](../Inbox/2026-04-27--learning-human-intention-priors-from-large-scale-human-demonstrations-for-robotic-manipulation.md): The paper abstract describes learning human-intention priors from reconstructed large-scale human demonstrations. - [$M^2$-VLA: Boosting Vision-Language Models for Generalizable Manipulation via Layer Mixture and Meta-Skills](../Inbox/2026-04-27--m-2-vla-boosting-vision-language-models-for-generalizable-manipulation-via-layer-mixture-and-meta-skills.md): $M^2$-VLA reports gains under rephrased instructions and novel-object tests using frozen VLM features and retrieved meta-skills.