psifx -- Psychological and Social Interactions Feature Extraction Package
Abstract
psifx is a plug-and-play multi-modal feature extraction toolkit, aiming to facilitate and democratize the use of state-of-the-art machine learning techniques for human sciences research. It is motivated by a need (a) to automate and standardize data annotation processes that typically require expensive, lengthy, and inconsistent human labour; (b) to develop and distribute open-source community-driven psychology research software; and (c) to enable large-scale access and ease of use for non-expert users. The framework contains an array of tools for tasks such as speaker diarization, closed-caption transcription and translation from audio; body, hand, and facial pose estimation and gaze tracking with multi-person tracking from video; and interactive textual feature extraction supported by large language models. The package has been designed with a modular and task-oriented approach, enabling the community to add or update new tools easily. This combination creates new opportunities for in-depth study of real-time behavioral phenomena in psychological and social science research.
Cite
@article{arxiv.2407.10266,
title = {psifx -- Psychological and Social Interactions Feature Extraction Package},
author = {Guillaume Rochette and Mathieu Rochat and Nizar Michaud and Matthew J. Vowels},
journal= {arXiv preprint arXiv:2407.10266},
year = {2026}
}