English

Action-grounded tissue affordance enables anticipatory auto-framing that lowers surgeon cognitive workload during laparoscopic surgery

Computer Vision and Pattern Recognition 2026-08-03 v1 Artificial Intelligence

Abstract

Computational attention models could help surgeons manage the visual demands of laparoscopy, but they require dense spatial labels that are difficult to obtain because surgical intent is highly specialized and tacit. Here, we introduce DiffeoAfford, an action-grounded tissue affordance framework that retrospectively derives visual attention supervision from completed surgical procedures. By combining diffeomorphism-constrained tissue tracking with instrument trajectory analysis, DiffeoAfford generates affordance hotspot labels without manual per-frame annotation. A real-time prediction model trained on these labels anticipates relevant surgical regions and enables AffordView, an assistive auto-framing system for laparoscopic visualization. The proposed framework aligns with expert annotations and intraoperative surgeon gaze, and reduces surgeon cognitive workload during real-world evaluations using subjective, physiological, and behavioral measures.

Keywords

Cite

@article{arxiv.2608.02471,
  title  = {Action-grounded tissue affordance enables anticipatory auto-framing that lowers surgeon cognitive workload during laparoscopic surgery},
  author = {Jiayu Gu and Yiwei Wang and Jie Zhang and Guojun Cao and Keshen Lyu and Song Zhou and Yimeng Chen and Haorui Wang and Qingmin Feng and Shenchao Shi and Huan Zhao and Wenbin Chen and Caihua Xiong and Chidan Wan and Jing Samantha Pan and Xiong Cai and Han Ding},
  journal= {arXiv preprint arXiv:2608.02471},
  year   = {2026}
}

Comments

Preprint. 54 pages, including supplementary information and 7 main figures