Source note
Grounding Hierarchical Vision-Language-Action Models Through Explicit Language-Action Alignment
Vision Language ActionHierarchical PolicyLanguage Action AlignmentPreference LearningRobot Transparency
Summary
GPLA trains a hierarchical vision-language-action model to make its generated subtask language match the robot's visual scene and action trajectory more closely. It does this with a learned grounding scorer and preference tuning, aiming to reduce the need for manual subtask annotations.
Problem
- Hierarchical VLA models can output both language substeps and robot actions, but their training usually does not force those two outputs to agree with each other.
- This matters for robot transparency and human-robot collaboration: a robot explanation is only useful if the described subtask matches what the robot sees and does.
- Existing evaluation often focuses on task success or action accuracy, while missing whether intermediate language is actually grounded in observation and behavior.
Approach
- The paper builds a hierarchical VLA with a high-level Gemma-3-4B-IT VLM that generates low-level instructions and a low-level SmolVLA that predicts 8-step action trajectories on LanguageTable.
- It trains a separate grounding model that maps vision-action pairs and text into a shared embedding space. The model uses frozen SigLIP 2 vision/text encoders, a small transformer for actions, and FiLM layers to condition vision features on actions.
- The grounding model is trained with symmetric InfoNCE contrastive loss plus a diversity regularizer, so matched language and vision-action pairs score higher than mismatched pairs.
- During GPLA training, the hierarchical VLA samples multiple language-action candidates for the same input, the grounding model ranks them, and the highest-scoring and lowest-scoring outputs form chosen/rejected preference pairs.
- The high-level VLM is then updated with SimPO preference optimization so it generates subtask descriptions that better align with the resulting actions and scene, without requiring new manual preference labels.
Results
- On LanguageTable, supervised training gives the best text-overlap metrics for low-level instruction generation: BLEU 0.111 ± 0.05, ROUGE 0.405 ± 0.12, METEOR 0.313 ± 0.12, BERTScore 0.984 ± 0.00.
- GPLA variants score lower on token-overlap text metrics, for example GPLA with the action-conditioned grounding model reaches BLEU 0.063 ± 0.05, ROUGE 0.300 ± 0.12, METEOR 0.218 ± 0.12, BERTScore 0.980 ± 0.00.
- Action quality stays close to the supervised baseline. Supervised has MSE 0.046 ± 0.02 and MAE 0.164 ± 0.04, while GPLA (action-conditioned) has MSE 0.045 ± 0.02 and MAE 0.163 ± 0.04.
- GPLA therefore claims action performance comparable to fully supervised fine-tuning while using preference pairs generated by a learned grounding score instead of costly extra annotations.
- Among GPLA variants, CLIP, SigLIP 2, and the action-conditioned grounding model produce very similar trajectory metrics; the paper's main claim is the explicit language-action grounding mechanism rather than a large raw performance jump.
- The paper also claims the action-conditioned grounding model yields a better shared embedding space for language and vision-action alignment than off-the-shelf vision-language encoders, based on embedding-space analysis and qualitative examples.