TRAFA: Anticipating User Actions to Reduce Errors in Procedural Tasks with Predictive Feedback
摘要
Interactive assistance systems typically provide feedback after an action has been completed, supporting error recovery but not preventing the error itself. We present TRAFA, a real-time predictive feedback system for procedural tasks that intervenes before errors are committed. TRAFA operationalizes predictive feedback through a Track-Forecast-Act framework that tracks hand and object state, forecasts user motion conditioned on scene context, and triggers feedback when a predicted action is likely to violate task constraints. We instantiate this pipeline in a sequential assembly setting and evaluate it through both technical benchmarking and a controlled user study against conventional reactive feedback. Our results show that predictive feedback improves task accuracy and efficiency while maintaining a comparable number of feedback events. These findings position feedback timing as a key dimension in system design and show how real-time anticipation can be integrated into interactive systems to prevent errors before they occur.
引用
@article{arxiv.2605.24526,
title = {TRAFA: Anticipating User Actions to Reduce Errors in Procedural Tasks with Predictive Feedback},
author = {Sassan Mokhtar and Lars Doorenbos and Fatemeh Jabbari and Marius Bock and Dominik Bach and Juergen Gall},
journal= {arXiv preprint arXiv:2605.24526},
year = {2026}
}