Detecting agreement in multi-party dialogue: evaluating speaker diarisation versus a procedural baseline to enhance user engagement
Abstract
Conversational agents participating in multi-party interactions face significant challenges in dialogue state tracking, since the identity of the speaker adds significant contextual meaning. It is common to utilise diarisation models to identify the speaker. However, it is not clear if these are accurate enough to correctly identify specific conversational events such as agreement or disagreement during a real-time interaction. This study uses a cooperative quiz, where the conversational agent acts as quiz-show host, to determine whether diarisation or a frequency-and-proximity-based method is more accurate at determining agreement, and whether this translates to feelings of engagement from the players. Experimental results show that our procedural system was more engaging to players, and was more accurate at detecting agreement, reaching an average accuracy of 0.44 compared to 0.28 for the diarised system.
Keywords
Cite
@article{arxiv.2311.03021,
title = {Detecting agreement in multi-party dialogue: evaluating speaker diarisation versus a procedural baseline to enhance user engagement},
author = {Angus Addlesee and Daniel Denley and Andy Edmondson and Nancie Gunson and Daniel Hernández Garcia and Alexandre Kha and Oliver Lemon and James Ndubuisi and Neil O'Reilly and Lia Perochaud and Raphaël Valeri and Miebaka Worika},
journal= {arXiv preprint arXiv:2311.03021},
year = {2023}
}
Comments
Proceedings of the workshop on advancing GROup UNderstanding and robots aDaptive behaviour (GROUND), 2023