Source note
VLA Foundry: A Unified Framework for Training Vision-Language-Action Models
Summary
VLA Foundry is an open-source codebase for training language, vision-language, and vision-language-action models in one stack. The paper’s main claim is that this unified pipeline makes VLA research easier to control and that stronger VLM backbones, such as Qwen3-VL, improve downstream robot policy performance.
Problem
- Open VLA projects often focus on the final action-training stage and rely on separate pretraining pipelines, which makes backbone, data-mixture, and recipe studies hard to run cleanly.
- Robotics data is scarce compared with text and image-text data, so upstream LLM and VLM training choices can strongly affect downstream robot performance.
- Researchers need one system that supports full-pipeline experiments, modular swaps of backbones and datasets, and reproducible large-scale training.
Approach
- The framework uses one shared codebase for LLM, VLM, and VLA training, with YAML/dataclass configuration, pluggable registries, shared training loops, and support for both from-scratch training and Hugging Face backbones.
- It standardizes multimodal data handling: text, image-caption, and robotics datasets can be mixed with weighted sampling, and robotics data gets dedicated normalization, action chunking, pose handling, and preprocessing into WebDataset shards.
- For scale, it supports distributed training with DDP/FSDP2, mixed precision, gradient checkpointing, and multi-node runs benchmarked up to 128 GPUs across 16 nodes.
- The paper demonstrates the framework with two model families: Foundry-VLA-1.7B trained through an LLM→VLM→VLA pipeline from scratch, and Foundry-Qwen3VLA-2.1B-MT built on a pretrained Qwen3-VL 2B backbone.
- The VLA model adds an observation token to the VLM sequence and feeds its hidden states into a 325M-parameter flow-transformer action head trained with flow matching to denoise action sequences.
Results
- The from-scratch LLM is a 1.2B-parameter model trained on 500M samples / 1T tokens from DCLM. On standard multiple-choice benchmarks, the 1T-token checkpoint reports HellaSwag 66.7, MMLU 26.6, ARC-e 71.7, ARC-c 39.3, PIQA 77.5, WinoGrande 62.6, OpenBookQA 40.8, and BoolQ 65.4.
- The from-scratch VLM combines that LLM with an 86M-parameter ViT and is trained on 200M DataCompDR-1B samples. On COCO_VAL captioning, the 200M-sample checkpoint reports BLEU-1 58.64, BLEU-2 38.62, BLEU-3 24.49, BLEU-4 15.57, ROUGE-L 38.17, and CIDEr 55.14.
- The VLA stack uses a 325M-parameter transformer action head on top of the VLM, giving a full from-scratch VLA model of 1.7B parameters; the Qwen-based multitask model is 2.1B parameters.
- The evaluation benchmark, lbm_eval_oss, contains 49 tabletop bimanual manipulation tasks in Drake simulation.
- The abstract claims that, in nominal closed-loop evaluation on LBM Eval, the fully open from-scratch model is on par with the authors’ prior closed-source work, and replacing the backbone with Qwen3-VL beats their baseline by a wide margin.
- The provided excerpt does not include the main VLA closed-loop success-rate table, so the exact robot-policy improvement numbers over baseline are not available here.