Current dialogue research primarily studies pairwise (two-party) conversations, and does not address the everyday setting where more than two speakers converse together. In this work, we both collect and evaluate multi-party conversations to study this more general case. We use the LIGHT environment to construct grounded conversations, where each participant has an assigned character to role-play. We thus evaluate the ability of language models to act as one or more characters in such conversations. Models require two skills that pairwise-trained models appear to lack: (1) being able to decide when to talk; (2) producing coherent utterances grounded on multiple characters. We compare models trained on our new dataset to existing pairwise-trained dialogue models, as well as large language models with few-shot prompting. We find that our new dataset, MultiLIGHT, which we will publicly release, can help bring significant improvements in the group setting.
@article{arxiv.2304.13835,
title = {Multi-Party Chat: Conversational Agents in Group Settings with Humans and Models},
author = {Jimmy Wei and Kurt Shuster and Arthur Szlam and Jason Weston and Jack Urbanek and Mojtaba Komeili},
journal= {arXiv preprint arXiv:2304.13835},
year = {2023}
}