GuidedAttention: Interpretable and Correctable Visual Attention for OOD-Robust Robot Manipulation via Imitation Learning

Masaki Murooka; Ryoichi Nakajo; Keisuke Shirai; Tomohiro Motoda; Hanbit Oh; Ryo Hanai; Yukiyasu Domae

GuidedAttention: Interpretable and Correctable Visual Attention for OOD-Robust Robot Manipulation via Imitation Learning

Robotics 2026-07-23 v1

Authors: Masaki Murooka , Ryoichi Nakajo , Keisuke Shirai , Tomohiro Motoda , Hanbit Oh , Ryo Hanai , Yukiyasu Domae

Abstract

End-to-end visuomotor policies provide little opportunity for humans to understand or correct the policy's visual attention. We propose GuidedAttention, a visuomotor imitation learning framework that introduces interpretable and correctable visual attention as an explicit intermediate representation. Task-relevant attention keypoints are predicted from camera images and condition a diffusion-based action policy. Users can inspect and optionally correct selected keypoints once at rollout initialization, after which the corrected attention is automatically propagated throughout execution by a tracking module. Experiments in simulation and the real world demonstrate that GuidedAttention consistently improves robot manipulation performance, particularly under positional and appearance out-of-distribution (OOD) conditions.

Cite

@article{arxiv.2607.21049,
  title  = {GuidedAttention: Interpretable and Correctable Visual Attention for OOD-Robust Robot Manipulation via Imitation Learning},
  author = {Masaki Murooka and Ryoichi Nakajo and Keisuke Shirai and Tomohiro Motoda and Hanbit Oh and Ryo Hanai and Yukiyasu Domae},
  journal= {arXiv preprint arXiv:2607.21049},
  year   = {2026}
}