Tactile-WAM: Touch-Aware World Action Model with Tactile Asymmetric Attention
Abstract
World Action Models (WAMs) generate actions together with predicted futures, offering a powerful interface for robot decision making. In contact-rich manipulation, however, visually plausible futures can be physically incomplete: insertion, assembly, search, and reorientation often depend on slip, jamming, contact normals, or small alignment errors that are weakly visible or hidden in RGB. A natural solution is to predict future tactile states, however, we identify tactile pollution, a failure mode where unconstrained tactile-token injection degrades video and action prediction by forcing a visual dynamics model to absorb sparse, local, event-driven contact signals. To address this, we propose Tactile-WAM, a touch-aware WAM with a Tactile Asymmetric Attention Mechanism (TAAM). TAAM combines a VideoClean mask, which blocks video-query access to tactile key/value tokens while preserving action-query access, with a touch-aware bias for action attention. The VideoClean mask protects visual prediction while keeping contact information available for action generation; the touch-aware bias is derived from predicted touch changes and modulates action attention to tactile tokens during denoising. On ManiFeel, Tactile-WAM improves the mean success rate by 38.9% overall and by 86% on contact-rich tasks.
Keywords
Cite
@article{arxiv.2606.26663,
title = {Tactile-WAM: Touch-Aware World Action Model with Tactile Asymmetric Attention},
author = {Siyu Wu and Linjing You and Junjie Zhu and Yaozu Liu and Changhao Zhang and Jian Liu and Weiqiang Wang and Qi Li and Jituo Li and Hengshuang Zhao},
journal= {arXiv preprint arXiv:2606.26663},
year = {2026}
}
Comments
Submitted to RSS2026 WorkShop Tactile for FM