Touchable projection with structured light range cameras is a prolific medium for large interaction surfaces, affording multiple simultaneous users and simple, cheap setup. However robust touch detection in such projector-depth systems is difficult to achieve due to measurement noise. We propose a novel combination of surface touch detection and a deep network for hand pose estimation, which aids in detecting both on- and above-surface hand gestures, disambiguating multiple touch fingers, as well as recovering fingertip positions in face of noisy input. We present the details of our GPU-accelerated system and an evaluation of its performance, as well as applications such as an enhanced virtual keyboard that utilizes the added features.
@article{arxiv.1812.11090,
title = {Enhanced Touchable Projector-depth System with Deep Hand Pose Estimation},
author = {Zhi Chai and Roy Shilkrot},
journal= {arXiv preprint arXiv:1812.11090},
year = {2018}
}