English

Adapt as You Say: Online Interactive Bimanual Skill Adaptation via Human Language Feedback

Robotics 2026-03-30 v1

Abstract

Developing general-purpose robots capable of autonomously operating in human living environments requires the ability to adapt to continuously evolving task conditions. However, adapting high-dimensional coordinated bimanual skills to novel task variations at deployment remains a fundamental challenge. In this work, we present BiSAIL (Bimanual Skill Adaptation via Interactive Language), a novel framework that enables zero-shot online adaptation of offline-learned bimanual skills through interactive language feedback. The key idea of BiSAIL is to adopt a hierarchical reason-then-modulate paradigm, which first infers generalized adaptation objectives from multimodal task variations, and then adapts bimanual motions via diffusion modulation to achieve the inferred objectives. Extensive real-robot experiments across six bimanual tasks and two dual-arm platforms demonstrate that BiSAIL significantly outperforms existing methods in human-in-the-loop adaptability, task generalization and cross-embodiment scalability. This work enables the development of adaptive bimanual assistants that can be flexibly customized by non-expert users via intuitive verbal corrections. Experimental videos and code are available at https://rip4kobe.github.io/BiSAIL/.

Keywords

Cite

@article{arxiv.2603.26466,
  title  = {Adapt as You Say: Online Interactive Bimanual Skill Adaptation via Human Language Feedback},
  author = {Zhuo Li and Dianxi Li and Tao Teng and Quentin Rouxel and Zhipeng Dong and Dennis Hong and Darwin Caldwell and Fei Chen},
  journal= {arXiv preprint arXiv:2603.26466},
  year   = {2026}
}

Comments

11 pages, 15 figures, submitted to IEEE TMECH