English

SANTS: A State-Adaptive Scheduler for World Action Models

Robotics 2026-05-28 v1

Abstract

World Action Models (WAMs) improve robot manipulation by using video-based future representations to condition action generation. In pixel-space WAMs, however, the best action condition is not necessarily the fully denoised video. Controlled denoising-depth scans show that video refinement can reduce action error up to a state-dependent point, after which the gain may saturate or even reverse when late predictions become less action-relevant or physically unreliable. This suggests that action generation should use a state-dependent point along the video noise trajectory rather than a fixed terminal denoising depth. We introduce State-Adaptive Noise Trajectory Scheduler (SANTS), a lightweight scheduler for video-to-action diffusion policies. At each video decision point, SANTS reads the current video-state representation and noise level, then jointly predicts a cumulative stopping hazard and a relative noise-progression ratio. SANTS is post-trained with a path-level reward computed after the frozen action branch generates the final action chunk, so the scheduler is optimized for downstream action quality rather than intermediate video fidelity, while redundant video-state updates are explicitly penalized. Experiments show that SANTS reaches 94.4%94.4\% overall success on RoboTwin 2.0 and 73.1%73.1\% average success across seven real-robot tasks, while reducing latency by 81.7%81.7\% and 79.0%79.0\% relative to full video denoising, respectively. These results indicate that adaptive selection along the video noise trajectory can preserve the control benefits of WAM-style future reasoning while removing much of its redundant inference cost.

Cite

@article{arxiv.2605.27947,
  title  = {SANTS: A State-Adaptive Scheduler for World Action Models},
  author = {Yirui Sun and Guangyu Zhuge and Keliang Liu and Jie Gu and Xinyu Bing and Zhongxue Gan and Chunxu Tian},
  journal= {arXiv preprint arXiv:2605.27947},
  year   = {2026}
}

Comments

17 pages, 5 figures, 8 tables. Project page: https://advanced-robotics-lab.github.io/SANTS/

R2 v1 2026-07-22T07:36:15.115Z