--- kind: ideas granularity: day period_start: '2026-07-16T00:00:00' period_end: '2026-07-17T00:00:00' run_id: aee749c8-06c4-43e2-acd7-f78c43e5b1f8 status: succeeded topics: - robot learning - vision-language-action models - world models - real-time control - robustness evaluation tags: - recoleta/ideas - topic/robot-learning - topic/vision-language-action-models - topic/world-models - topic/real-time-control - topic/robustness-evaluation language_code: en pass_output_id: 361 pass_kind: trend_ideas upstream_pass_output_id: 360 upstream_pass_kind: trend_synthesis --- # Targeted evaluation and temporal supervision for robot deployment ## Summary Robot deployment teams can use structured testing to find environmental failures without erasing task-relevant perception, apply predictive tactile supervision to memory-bearing policy states, and combine simulated rollouts with selectively chosen hardware trials. Each change addresses a specific gap between benchmark performance and reliable physical operation. ## Active lighting tests that preserve color-dependent capability Robot QA teams should include spotlight hue, intensity, position, and beam angle as factors in active real-world evaluation, while separating tasks that require color from those that do not. FLARE shows why failure discovery alone is insufficient: broad color augmentation survived attacks partly by teaching policies to ignore color, cutting benign success on a real color-dependent task to 47.5%. A factor-based surrogate could instead select informative lighting configurations and track both attack robustness and retained color discrimination, reducing physical trials without rewarding this shortcut. The cheapest check is to add grayscale and color-swap diagnostics to an existing active-testing run, then compare the discovered failure regions with uniform-random lighting tests under the same hardware budget. ### Sources - [Lights, Camera, Malfunction: When Illumination Robustness Leaves VLA Models Blind to Color](../Inbox/2026-07-16--lights-camera-malfunction-when-illumination-robustness-leaves-vla-models-blind-to-color.md): Optimized physical lighting reduced baseline task success to zero, while naive augmentation preserved grayscale performance but harmed benign color-dependent behavior. - [Active Real-World Factor-Based Evaluation for Generalist Robot Policies](../Inbox/2026-07-16--active-real-world-factor-based-evaluation-for-generalist-robot-policies.md): Structured active evaluation across real-world factors typically saved 20–40% of physical trials compared with random testing over 2,331 evaluations. ## Future-tactile supervision for long-context assembly memory Teams building insertion and multi-stage assembly policies should test future-tactile prediction on the recurrent state that carries long interaction history, rather than attaching it only to vision features or final motor outputs. RoboTTT shows that fast weights can compress up to 8K visuomotor timesteps without latency growing with context; the tactile study finds that future contact is most useful when it supervises intermediate action-expert features and reports 74% average success across five real tasks. Together, they suggest training the memory-bearing action representation to retain contact consequences such as slip, resistance, and alignment, then removing the tactile predictor at deployment. A focused evaluation should compare short and long context on tasks where an early contact event changes a later correction—such as a partial insertion followed by regrasping—and probe whether the fast-weight state predicts future tactile embeddings better than the final action state. ### Sources - [RoboTTT: Context Scaling for Robot Policies](../Inbox/2026-07-16--robottt-context-scaling-for-robot-policies.md): RoboTTT compresses histories of up to 8K timesteps into fast weights while keeping inference latency constant with context length. - [Representation-Aligned Tactile Grounding for Contact-Rich Robotic Manipulation](../Inbox/2026-07-16--representation-aligned-tactile-grounding-for-contact-rich-robotic-manipulation.md): Future tactile states were most predictable from intermediate action-expert features; the training-only predictor improved contact-rich manipulation without inference-time computation. ## World-model pre-screening for real-hardware active evaluation Robot evaluation teams can use a fast action-conditioned world model to pre-screen task-factor configurations, then spend physical trials where simulated outcomes are uncertain or disagree with the real-hardware surrogate. DriftWorld generates futures in one forward pass at more than 30 fps and its rollout scores correlated with ground-truth policy performance at 0.925–0.992 on three reported tasks. Active factor-based evaluation, however, treats real hardware as necessary because simulator discrepancies can hide failures and shows that sequential selection already saves 20–40% of trials. A multi-fidelity evaluator should therefore use simulated rollouts to rank pose, viewpoint, and table-height combinations, while updating a discrepancy model from selected physical trials rather than treating generated video as ground truth. The decision-changing check is a fixed-budget comparison against hardware-only active testing: measure error in the estimated performance surface and the number of distinct real failure regions found. If simulation-guided selection repeatedly overlooks failures that the hardware-only method finds, the world model should remain an offline ranking aid rather than drive test allocation. ### Sources - [DriftWorld: Fast World Modeling through Drifting](../Inbox/2026-07-16--driftworld-fast-world-modeling-through-drifting.md): DriftWorld supports offline policy ranking, with rollout-based scores reported to correlate with ground truth at up to 0.99. - [Active Real-World Factor-Based Evaluation for Generalist Robot Policies](../Inbox/2026-07-16--active-real-world-factor-based-evaluation-for-generalist-robot-policies.md): Real-world factor evaluation covers combinatorial deployment conditions and reports 20–40% fewer trials than random testing.