We present a new framework for vision-based estimation of calorific expenditure from RGB-D data - the first that is validated on physical gas exchange measurements and applied to daily living scenarios. Deriving a person's energy expenditure from sensors is an important tool in tracking physical activity levels for health and lifestyle monitoring. Most existing methods use metabolic lookup tables (METs) for a manual estimate or systems with inertial sensors which ultimately require users to wear devices. In contrast, the proposed pose-invariant and individual-independent vision framework allows for a remote estimation of calorific expenditure. We introduce, and evaluate our approach on, a new dataset called SPHERE-calorie, for which visual estimates can be compared against simultaneously obtained, indirect calorimetry measures based on gas exchange. % based on per breath gas exchange. We conclude from our experiments that the proposed vision pipeline is suitable for home monitoring in a controlled environment, with calorific expenditure estimates above accuracy levels of commonly used manual estimations via METs. With the dataset released, our work establishes a baseline for future research for this little-explored area of computer vision.
@article{arxiv.1607.08196,
title = {Calorie Counter: RGB-Depth Visual Estimation of Energy Expenditure at Home},
author = {Lili Tao and Tilo Burghardt and Majid Mirmehdi and Dima Damen and Ashley Cooper and Sion Hannuna and Massimo Camplani and Adeline Paiement and Ian Craddock},
journal= {arXiv preprint arXiv:1607.08196},
year = {2016}
}