We present an on-device real-time hand gesture recognition (HGR) system, which detects a set of predefined static gestures from a single RGB camera. The system consists of two parts: a hand skeleton tracker and a gesture classifier. We use MediaPipe Hands as the basis of the hand skeleton tracker, improve the keypoint accuracy, and add the estimation of 3D keypoints in a world metric space. We create two different gesture classifiers, one based on heuristics and the other using neural networks (NN).
@article{arxiv.2111.00038,
title = {On-device Real-time Hand Gesture Recognition},
author = {George Sung and Kanstantsin Sokal and Esha Uboweja and Valentin Bazarevsky and Jonathan Baccash and Eduard Gabriel Bazavan and Chuo-Ling Chang and Matthias Grundmann},
journal= {arXiv preprint arXiv:2111.00038},
year = {2021}
}
Comments
5 pages, 6 figures; ICCV Workshop on Computer Vision for Augmented and Virtual Reality, Montreal, Canada, 2021