--- kind: ideas granularity: day period_start: '2026-06-05T00:00:00' period_end: '2026-06-06T00:00:00' run_id: fb2f6c8f-61ef-4708-a48e-b9d412b25db6 status: succeeded topics: - robotics - vision-language-action - action representation - policy adaptation - long-horizon control - edge deployment tags: - recoleta/ideas - topic/robotics - topic/vision-language-action - topic/action-representation - topic/policy-adaptation - topic/long-horizon-control - topic/edge-deployment language_code: en pass_output_id: 259 pass_kind: trend_ideas upstream_pass_output_id: 258 upstream_pass_kind: trend_synthesis --- # VLA action interface evaluation ## Summary Robot manipulation teams now have concrete tests to run at the action interface: swap point decoders for voxel heatmaps, profile VLA latency by token and action-generation cost, and generate task LoRA adapters from a prompt plus short video when action labels are unavailable. The useful checks are narrow: matched-budget LIBERO and Franka trials for action heads, 10 Hz closed-loop profiling on edge hardware, and held-out task adaptation with clear failure reporting on object and long-horizon tasks. ## Voxel heatmap action heads for VLA manipulation policies Teams training OpenVLA-OFT, π0.5, or similar VLA policies should test the action head as a replaceable component. ActionMap keeps the backbone fixed and replaces single-point action prediction with voxel heatmaps for translation, rotation, and gripper state. The training target is a soft Gaussian blob over the action grid, and inference recovers a continuous command with top-k soft argmax. The practical test is small: run the same training budget with the native decoder and with a voxel heatmap head on LIBERO-Spatial and LIBERO-Long, then repeat on one real Franka pick or place task with grasp-position error logged. ActionMap reports a LIBERO four-suite gain for OpenVLA-OFT from 89.1% to 97.3%, a 10% data LIBERO-Spatial gain from 67.2% to 93.2%, and real Franka full-data trials rising from 7/30 to 20/30. Those numbers make the decoder swap a credible first experiment for teams seeing millimeter-scale end-effector errors or low-data failures. ### Sources - [ActionMap: Robot Policy Learning via Voxel Action Heatmap](../Inbox/2026-06-05--actionmap-robot-policy-learning-via-voxel-action-heatmap.md): ActionMap summary gives the drop-in voxel heatmap action head, LIBERO gains, low-data result, and Franka trial counts. - [ActionMap: Robot Policy Learning via Voxel Action Heatmap](../Inbox/2026-06-05--actionmap-robot-policy-learning-via-voxel-action-heatmap.md): The paper abstract describes the action decoder as the component converting VLA hidden states into continuous control and names the single-point predictor limitation. ## Edge VLA latency audits centered on visual tokens and action generation Robot teams trying to run closed-loop VLA control on onboard hardware should profile the VLM backbone, visual-token count, and Action Expert before changing the controller. RhinoVLA’s report gives a concrete latency target and breakdown: it reaches 11.69 Hz end-to-end inference on the Huixi R1 edge SoC, above a 10 Hz control target, while its π0.5 analysis on Jetson AGX Orin puts more than 90% of runtime in the VLM backbone and Action Expert. A useful build is a deployment harness that records image-token count, VLM projection time, Action Expert time, and end-to-end command rate for each camera configuration. RhinoVLA’s design also points to the data-interface work needed for mixed robot fleets: a View Registry for camera role and modality, a shared 72D state-action slot space, masks for missing dimensions, and robot-instance LoRA modules. The report does not give detailed task-level scores in the excerpt, so the first adoption check should pair the latency target with a fixed manipulation success suite on the target robot. ### Sources - [RhinoVLA Technical Report](../Inbox/2026-06-05--rhinovla-technical-report.md): RhinoVLA summary gives the 11.69 Hz result, 10 Hz target, Jetson latency breakdown, visual-token comparison, and cross-robot interface design. - [RhinoVLA Technical Report](../Inbox/2026-06-05--rhinovla-technical-report.md): The content chunk explains why VLM backbone and Action Expert dominate latency and how visual-token count affects GEMM-heavy projection cost. ## Generated LoRA adapters from a short task video for new manipulation tasks Labs that already maintain task-specific LoRA adapters can test a generated-adapter workflow for new manipulation tasks with no target action labels. WIZARD trains expert LoRAs on known tasks, encodes a new task from its language instruction and short demonstration video, and predicts the LoRA weights for a frozen VLA policy in one forward pass. The robot then runs the adapted policy without gradient updates at test time. The workflow fits task onboarding where a human can provide a short video but cannot provide synchronized robot actions. The first benchmark should separate spatial rearrangement tasks from object-identity changes. WIZARD reaches 0.40 average success on held-out LIBERO-Spatial versus 0.19 for MT-VLA with π0.5 and 0.02 for nearest-neighbor adapter retrieval, but LIBERO-Object remains low at 0.03 and full-task zero-shot completion on LIBERO-10 is 0.00 in the excerpt. That boundary matters for deployment planning: use the method first where task geometry changes more than the object distribution. ### Sources - [Robotic Policy Adaptation via Weight-Space Meta-Learning](../Inbox/2026-06-05--robotic-policy-adaptation-via-weight-space-meta-learning.md): WIZARD summary gives the prompt-plus-video LoRA generation method, no action-label and no fine-tuning setup, and held-out LIBERO results with limitations. - [Robotic Policy Adaptation via Weight-Space Meta-Learning](../Inbox/2026-06-05--robotic-policy-adaptation-via-weight-space-meta-learning.md): The content chunk states the deployment cost of task-specific fine-tuning and action-labeled demonstrations.