Real-time and Continuous Turn-taking Prediction Using Voice Activity Projection
Computation and Language
2024-01-11 v1 Human-Computer Interaction
Sound
Audio and Speech Processing
Abstract
A demonstration of a real-time and continuous turn-taking prediction system is presented. The system is based on a voice activity projection (VAP) model, which directly maps dialogue stereo audio to future voice activities. The VAP model includes contrastive predictive coding (CPC) and self-attention transformers, followed by a cross-attention transformer. We examine the effect of the input context audio length and demonstrate that the proposed system can operate in real-time with CPU settings, with minimal performance degradation.
Keywords
Cite
@article{arxiv.2401.04868,
title = {Real-time and Continuous Turn-taking Prediction Using Voice Activity Projection},
author = {Koji Inoue and Bing'er Jiang and Erik Ekstedt and Tatsuya Kawahara and Gabriel Skantze},
journal= {arXiv preprint arXiv:2401.04868},
year = {2024}
}
Comments
This paper has been accepted for presentation at International Workshop on Spoken Dialogue Systems Technology 2024 (IWSDS 2024) and represents the author's version of the work