We present IMPACT-HOI, a mixed-initiative framework for annotating egocentric procedural video by constructing structured event graphs for Human-Object Interactions (HOI), motivated by the need for high-quality structured supervision for learning robot manipulation from human demonstration. IMPACT-HOI frames this task as the incremental resolution of a partially specified, onset-anchored event state. A trust-calibrated controller selects among direct queries, human-confirmed suggestions, and conservative completions based on empirical annotator behavior and evidence quality. A risk-bounded execution protocol, utilizing atomic rollback, ensures that human-confirmed decisions are preserved against conflicting automated updates. A user study with 9 participants shows a 13.5% reduction in manual annotation actions, a 46.67% event match rate, and zero confirmed-field violations under the studied protocol. The code will be made publicly available at https://github.com/541741106/IMPACT_HOI.
@article{arxiv.2605.01666,
title = {IMPACT-HOI: Supervisory Control for Onset-Anchored Partial HOI Event Construction},
author = {Haoshen Zhang and Di Wen and Kunyu Peng and David Schneider and Zeyun Zhong and Alexander Jaus and Zdravko Marinov and Jiale Wei and Ruiping Liu and Junwei Zheng and Yufan Chen and Yufeng Zhang and Yuanhao Luo and Lei Qi and Rainer Stiefelhagen},
journal= {arXiv preprint arXiv:2605.01666},
year = {2026}
}
Comments
8 pages, 2 figures. Code is available at https://github.com/541741106/IMPACT_HOI