Hand gesture classification using high-quality structured data such as videos, images, and hand skeletons is a well-explored problem in computer vision. Leveraging low-power, cost-effective biosignals, e.g. surface electromyography (sEMG), allows for continuous gesture prediction on wearables. In this paper, we demonstrate that learning representations from weak-modality data that are aligned with those from structured, high-quality data can improve representation quality and enables zero-shot classification. Specifically, we propose a Contrastive Pose-EMG Pre-training (CPEP) framework to align EMG and pose representations, where we learn an EMG encoder that produces high-quality and pose-informative representations. We assess the gesture classification performance of our model through linear probing and zero-shot setups. Our model outperforms emg2pose benchmark models by up to 21% on in-distribution gesture classification and 72% on unseen (out-of-distribution) gesture classification.
@article{arxiv.2509.04699,
title = {CPEP: Contrastive Pose-EMG Pre-training Enhances Gesture Generalization on EMG Signals},
author = {Wenhui Cui and Christopher Sandino and Hadi Pouransari and Ran Liu and Juri Minxha and Ellen Zippi and Aman Verma and Anna Sedlackova and Erdrin Azemi and Behrooz Mahasseni},
journal= {arXiv preprint arXiv:2509.04699},
year = {2025}
}
Comments
Accepted by 39th Conference on Neural Information Processing Systems (NeurIPS 2025) Workshop: Foundation Models for the Brain and Body