Real-time cognitive workload monitoring is crucial in safety-critical environments, yet established measures are intrusive, expensive, or lack temporal resolution. We tested whether facial movement dynamics from a standard webcam could provide a low-cost alternative. Seventy-two participants completed a multitasking simulation (OpenMATB) under varied load while facial keypoints were tracked via OpenPose. Linear kinematics (velocity, acceleration, displacement) and recurrence quantification features were extracted. Increasing load altered dynamics across timescales: movement magnitudes rose, temporal organisation fragmented then reorganised into complex patterns, and eye-head coordination weakened. Random forest classifiers trained on pose kinematics outperformed task performance metrics (85% vs. 55% accuracy) but generalised poorly across participants (43% vs. 33% chance). Participant-specific models reached 50% accuracy with minimal calibration (2 minutes per condition), improving continuously to 73% without plateau. Facial movement dynamics sensitively track workload with brief calibration, enabling adaptive interfaces using commodity cameras, though individual differences limit cross-participant generalisation.
@article{arxiv.2603.17767,
title = {Facial Movement Dynamics Reveal Workload During Complex Multitasking},
author = {Carter Sale and Melissa N. Stolar and Gaurav Patil and Michael J. Gostelow and Julia Wallier and Margaret C. Macpherson and Jan-Louis Kruger and Mark Dras and Simon G. Hosking and Rachel W. Kallen and Michael J. Richardson},
journal= {arXiv preprint arXiv:2603.17767},
year = {2026}
}
Comments
26 pages, 7 figures, under review at Royal Society Open Science