Source note
Long-Term Memory for VLA-based Agents in Open-World Task Execution
Summary
ChemBot targets long-horizon chemical lab execution with a VLA agent that can plan, act, track progress, and reuse past successful trajectories. The paper’s main claim is that adding dual-layer memory, closed-loop subtask planning, and a progress-aware VLA improves decomposition quality and real-world task success over VLA baselines.
Problem
- VLA systems can map language and vision to robot actions, but they often struggle with long, state-dependent lab protocols where errors compound across many steps.
- Existing systems usually separate planning from execution and do not keep a persistent memory of successful runs, so the robot repeats trial-and-error instead of reusing what worked.
- This matters in chemical labs because tasks are multi-stage, safety-sensitive, and require precise switching between subtasks based on the current physical state.
Approach
- The system, called ChemBot, uses a two-level design: an LLM-based agent plans the task as atomic subtasks, and a VLA policy called Skill-VLA executes each subtask on the robot.
- A Scene Describer converts the current bench image into a structured dashboard with objects, states, affordances, and spatial relations, so planning uses grounded scene information instead of raw images alone.
- Planning runs in a closed loop with a Subtask Reasoner that proposes one atomic step at a time and a Reflector that checks feasibility, redundancy, and sequence validity before adding the step to the queue.
- The memory system has short-term memory for current scene state, tool outputs, and task progress, plus long-term episodic memory that stores prior successful trajectories for semantic retrieval in later experiments.
- Skill-VLA builds on GR00T and adds a progress head that predicts a scalar subtask completion value in �[0,1]. When progress crosses a threshold, the agent ends the current skill and starts the next one. The paper also adds asynchronous future-state inference to reduce action discontinuities.
Results
- For chemical task decomposition, among tested base VLMs, Qwen3-VL-Plus had the best text/structure scores with Edit Distance 0.733, BERTScore F1 0.744, ROUGE-1 0.542, ROUGE-2 0.366, and ROUGE-L 0.471. Qwen3-VL-Flash was faster at 98.8 s total and 16 s/step versus 298.8 s and 42 s/step for Qwen3-VL-Plus.
- In ChemBot ablations with Qwen3-VL-Flash, the full model reached Edit Distance 0.766, BERTScore F1 0.694, ROUGE-L 0.324, 98.8 s runtime, and 22,401 tokens. Removing the Scene Describer hurt decomposition quality most, raising Edit Distance to 0.915 and dropping BERTScore F1 to 0.556.
- Removing the Subtask Chain caused the largest structural collapse in ablations, with Edit Distance 1.195 versus 0.766 for the full model. Removing the Reflector gave Edit Distance 0.881. Removing Memory increased token length from 22,401 to 28,064, suggesting memory mainly reduces context load while keeping quality close to the full model.
- The robot dataset used to train Skill-VLA contains 5,459 expert trajectories, averaging 872.2 frames, with 51,580 atomic action segments across 12 categories of chemical procedures.
- Real-world evaluation used a UR3 robot, 16 trials per task, and three multi-step experiments: Precipitation (11 steps), Heat and Dissolution (7 steps), and Neutralization (6 steps). The paper states that Skill-VLA with subtask-level training outperformed full-trajectory baselines (\pi_{0.5}) and GR00T on all three tasks.
- The excerpt does not provide the exact success-rate percentages from Fig. 8, so the strongest supported claim is qualitative: the full ChemBot strategy achieved higher real-world success rates, safety, and precision than full-trajectory VLA baselines, and the authors attribute the gain to subtask decomposition plus progress-based skill switching.