Physical VLA policy validation

A VLA lab can make a small physical benchmark part of every policy release: rebuild the same SO-101 arm setup, run a fixed set of manipulation tasks, and publish per-task success on in-distribution and out-of-distribution scenes. VLA-REPLICA gives enough detail for this workflow to be practical: an SO-101 6-DoF arm, RealSense D455 top camera, wrist webcam, 32 inch light box, commodity objects, camera overlay tools, AprilTag alignment, fixed lighting, reference images, and predefined placements. The reported setup cost is about $1050, and a new user assembled it in under an hour.

The useful check is simple: train or fine-tune on the same 500 expert demonstrations, run the 90 defined test scenes, and compare against reported baselines such as π₀.₅ at 0.54 average success on the 10 in-distribution tasks. This gives robotics groups a cheap physical gate before claiming that a VLA policy works outside simulation.

Manipulation teams working on dense boards, bins, or object layouts should require the policy to expose a target state before predicting actions. AVP is a concrete pattern: the VLM predicts visual primitives such as points, boxes, masks, or memory primitives, projects them into visual token space, and conditions a flow-matching action expert. The labels come from end-effector kinematics through camera calibration, so teams can create supervision without hand-labeling every target prompt.

This also gives developers a better failure log. When a run fails, they can inspect whether the model selected the wrong object, chose a bad placement region, or produced a poor motor sequence after a correct target. AVP reports 90.28% average success on Chinese chess manipulation versus 62.67% for π₀.₅, with 0.27 seconds per instruction. SOMA points to the same operational need for partially visible scenes: persistent object memory with semantic and 3D position data cut time-to-grasp across five out-of-vision tasks and raised average success to 28.3% in its real-world ablation.

Teams with RGB-D cameras can test whether geometry belongs inside the action path, not just in a perception preprocessor. PointACT gives one direct build pattern: encode colored point clouds with Point Transformer v3, let action tokens attend to local point windows at multiple scales, then combine those geometry-conditioned action tokens with VLM features. The reported LIBERO average is 96.0%, including a 17.9 point gain over SpatialVLA in the cited table.

GaussianDream offers a training-time variant for teams that want geometry supervision without extra inference modules. It decodes learned prefix tokens into current 3D Gaussian scenes and short-horizon future Gaussian motion during training, using RGB rendering, depth, and pseudo 3D scene-flow losses. At inference, the Gaussian heads are removed and the policy keeps only the learned prefix tokens. The paper reports 98.4% average success on LIBERO, 52.6% on RoboCasa Human-50, and 50.0% in real-robot evaluation. A useful adoption test is to compare grasp-point and spatial-relation failures before and after adding either point-action attention or training-time 3D supervision.