In the context of autonomous driving, where humans may need to take over in the event where the computer may issue a takeover request, a key step towards driving safety is the monitoring of the hands to ensure the driver is ready for such a request. This work, focuses on the first step of this process, which is to locate the hands. Such a system must work in real-time and under varying harsh lighting conditions. This paper introduces a fast ConvNet approach, based on the work of original work of OpenPose for full body joint estimation. The network is modified with fewer parameters and retrained using our own day-time naturalistic autonomous driving dataset to estimate joint and affinity heatmaps for driver & passenger's wrist and elbows, for a total of 8 joint classes and part affinity fields between each wrist-elbow pair. The approach runs real-time on real-world data at 40 fps on multiple drivers and passengers. The system is extensively evaluated both quantitatively and qualitatively, showing at least 95% detection performance on joint localization and arm-angle estimation.
@article{arxiv.1804.01176,
title = {Looking at Hands in Autonomous Vehicles: A ConvNet Approach using Part Affinity Fields},
author = {Kevan Yuen and Mohan M. Trivedi},
journal= {arXiv preprint arXiv:1804.01176},
year = {2018}
}
Comments
11 pages, 8 figures, 1 table. Submitted to "IEEE Transactions on Intelligent Vehicles" (under review)