Language-guided manipulation control

A practical baseline for many VLA teams is now a pretrained VLM with a small continuous MLP action head, evaluated under one fixed recipe across tasks. StarVLA-α reports 98.8 average on LIBERO, 88.2 on RoboTwin 2.0 clean*, and 53.8 on RoboCasa-GR1 with a minimal pipeline and no benchmark-specific tuning. In the same setup, the MLP head matches or beats heavier action heads: 53.8 on RoboCasa-GR1 versus 52.8 for a GR00T-style head and 48.9 for a diffusion-style head, and 76.0 on SimplerEnv Google VM versus 60.1 for FAST.

The workflow change is straightforward. Before adding robot pretraining, action tokenizers, diffusion heads, or extra interface features, teams can maintain a fixed baseline built from a strong pretrained VLM, raw RGB, language instruction, training-split action normalization, and a continuous MLP head. New components should enter only after they clear that baseline on the same backbone and data. The ablations support that discipline: extra robot pretraining helps one benchmark and hurts another, and common data-engineering additions shrink to small gains once task data is large.

A cheap check is to rerun one current internal model comparison with the backbone and dataset held constant, then compare only the action head and one added training trick. If the simple head stays within a few points or wins outright, the team can cut model complexity and keep evaluation cleaner.

Robotics teams working on instruction following can add a language-scored planning harness to test semantic generalization before training another end-to-end policy. Grounded World Model shows a concrete way to do this: predict future latent states for candidate action chunks, score each future against the text instruction in a shared vision-language embedding space, and execute the best-scoring chunk. On WISER, this reaches 0.87 test success with 0.92 train success, while the VLA baselines average 0.90 on train and 0.22 on test.

This matters for workflows where the robot has seen the motions before but fails when the wording or visual cue changes. WISER is built around that exact pressure. Named VLA baselines drop to 0.47 for InstructVLA, 0.40 for Wall-OSS, 0.26 for InternVLA-A1 and π0.5, and lower for others on the held-out test tasks. GWM-MPC also reaches 0.83 test success after transfer to xArm6, which suggests the scoring layer is useful across embodiments.

A buildable first step is a reranker over retrieved trajectories or policy proposals. Keep the current proposal generator, predict short-horizon outcomes, and rank them by cosine similarity to the instruction in a frozen multimodal embedding model. The first validation pass does not need full robot retraining. It needs a held-out instruction split with paraphrases, new referring expressions, and visual substitutions that preserve the required motion.

Affordance-heavy manipulation still needs a support layer between task text and trajectory generation. AffordSim shows the gap clearly. On 17 representative tasks, Pi 0.5 averages 61% success, but pouring and mug hanging remain much lower than easy grasping. Example tasks reach 93% on pick_banana, 43% on pour_cup_into_bowl, and 47% on hang_mug_on_rack. The generator-side ablation points to the missing piece: AnyGrasp averages 20% on trajectory generation, while VoxAfford reaches 61%, with large gains on affordance-sensitive tasks such as pour_into_cup at 63% versus 0%.

The concrete build is an affordance-conditioned grasp proposer for data generation and policy evaluation. Given a task phrase such as "graspable handle" or "pourable rim," predict 3D affordance regions on the object point cloud, sample grasps around those regions, and score them for both affordance contact and reachability. That is useful for teams training imitation policies in simulation, because physically valid grasps still produce task-wrong demonstrations on objects with handles, rims, hooks, and openings.

A cheap check is to pick five existing tasks that depend on object parts, regenerate demonstrations with affordance-conditioned grasps, and compare success against a generic grasp planner. The early readout should focus on pouring, hanging, and tool-use tasks, where part selection decides most of the outcome.