English
Related papers

Related papers: Improving Automated Evaluation of Open Domain Dial…

200 papers

Humans often refer to personal narratives, life experiences, and events to make a conversation more engaging and rich. While persona-grounded dialog models are able to generate responses that follow a given persona, they often miss out on…

Computation and Language · Computer Science 2021-06-17 Bodhisattwa Prasad Majumder , Taylor Berg-Kirkpatrick , Julian McAuley , Harsh Jhamtani

Aligning language models with human preferences through reinforcement learning from human feedback is crucial for their safe and effective deployment. The human preference is typically represented through comparison where one response is…

Machine Learning · Computer Science 2025-07-15 Hoang Anh Just , Ming Jin , Anit Sahu , Huy Phan , Ruoxi Jia

Automated generalisation has known important improvements these last few years. However, an issue that still deserves more study concerns the automatic evaluation of generalised data. Indeed, many automated generalisation systems require…

Human-Computer Interaction · Computer Science 2012-04-20 Patrick Taillandier , Julien Gaffuri

This paper suggests a direction of coreference resolution for online decoding on actively generated input such as dialogue, where the model accepts an utterance and its past context, then finds mentions in the current utterance as well as…

Computation and Language · Computer Science 2022-05-24 Liyan Xu , Jinho D. Choi

The training of task-oriented dialogue systems is often confronted with the lack of annotated data. In contrast to previous work which augments training data through expensive crowd-sourcing efforts, we propose four different automatic…

Computation and Language · Computer Science 2019-12-06 Jun Quan , Deyi Xiong

Open-domain dialog involves generating search queries that help obtain relevant knowledge for holding informative conversations. However, it can be challenging to determine what information to retrieve when the user is passive and does not…

Computation and Language · Computer Science 2023-10-24 Revanth Gangi Reddy , Hao Bai , Wentao Yao , Sharath Chandra Etagi Suresh , Heng Ji , ChengXiang Zhai

In this paper, a novel Generation-Evaluation framework is developed for multi-turn conversations with the objective of letting both participants know more about each other. For the sake of rational knowledge utilization and coherent…

Computation and Language · Computer Science 2019-06-04 Siqi Bao , Huang He , Fan Wang , Rongzhong Lian , Hua Wu

Our research is focused on making a human-like question answering system which can answer rationally. The distinguishing characteristic of our approach is that it will use automated common sense reasoning to truly "understand" dialogues,…

Artificial Intelligence · Computer Science 2019-09-19 Kinjal Basu

Conversational search aims to satisfy users' complex information needs via multiple-turn interactions. The key challenge lies in revealing real users' search intent from the context-dependent queries. Previous studies achieve conversational…

Information Retrieval · Computer Science 2025-11-13 Fengran Mo , Jinghan Zhang , Yuchen Hui , Jia Ao Sun , Zhichao Xu , Zhan Su , Jian-Yun Nie

Learning from free-text human feedback is essential for dialog systems, but annotated data is scarce and usually covers only a small fraction of error types known in conversational AI. Instead of collecting and annotating new datasets from…

Computation and Language · Computer Science 2023-10-25 Dominic Petrak , Nafise Sadat Moosavi , Ye Tian , Nikolai Rozanov , Iryna Gurevych

Encoder-decoder based neural architectures serve as the basis of state-of-the-art approaches in end-to-end open domain dialog systems. Since most of such systems are trained with a maximum likelihood~(MLE) objective they suffer from issues…

The emergence of Large Language Models (LLMs) as chat assistants capable of generating human-like conversations has amplified the need for robust evaluation methods, particularly for open-ended tasks. Conventional metrics such as EM and F1,…

Computation and Language · Computer Science 2025-11-12 Sher Badshah , Hassan Sajjad

Despite recent progress in open-domain dialogue evaluation, how to develop automatic metrics remains an open problem. We explore the potential of dialogue evaluation featuring dialog act information, which was hardly explicitly modeled in…

Computation and Language · Computer Science 2022-11-04 Jianqiao Zhao , Yanyang Li , Wanyu Du , Yangfeng Ji , Dong Yu , Michael R. Lyu , Liwei Wang

Chatbots are designed to carry out human-like conversations across different domains, such as general chit-chat, knowledge exchange, and persona-grounded conversations. To measure the quality of such conversational agents, a dialogue…

Computation and Language · Computer Science 2022-01-19 Chen Zhang , Luis Fernando D'Haro , Thomas Friedrichs , Haizhou Li

Retrieval-based conversational systems learn to rank response candidates for a given dialogue context by computing the similarity between their vector representations. However, training on a single textual form of the multi-turn context…

Computation and Language · Computer Science 2022-04-19 Lahari Poddar , Peiyao Wang , Julia Reinspach

Building dialog agents that can converse naturally with humans is a challenging yet intriguing problem of artificial intelligence. In open-domain human-computer conversation, where the conversational agent is expected to respond to human…

Artificial Intelligence · Computer Science 2018-02-13 Tom Young , Erik Cambria , Iti Chaturvedi , Minlie Huang , Hao Zhou , Subham Biswas

Open-domain human-computer conversation has attracted much attention in the field of NLP. Contrary to rule- or template-based domain-specific dialog systems, open-domain conversation usually requires data-driven approaches, which can be…

Computation and Language · Computer Science 2016-10-25 Yiping Song , Rui Yan , Xiang Li , Dongyan Zhao , Ming Zhang

In spoken dialogue systems, we aim to deploy artificial intelligence to build automated dialogue agents that can converse with humans. Dialogue systems are increasingly being designed to move beyond just imitating conversation and also…

Computation and Language · Computer Science 2021-11-03 Atharv Singh Patlan , Shiven Tripathi , Shubham Korde

Existing automatic evaluation metrics for open-domain dialogue response generation systems correlate poorly with human evaluation. We focus on evaluating response generation systems via response selection. To evaluate systems properly via…

Computation and Language · Computer Science 2020-04-30 Shiki Sato , Reina Akama , Hiroki Ouchi , Jun Suzuki , Kentaro Inui

In this work, we tackle a problem of speech emotion classification. One of the issues in the area of affective computation is that the amount of annotated data is very limited. On the other hand, the number of ways that the same emotion can…

Computation and Language · Computer Science 2018-04-02 Egor Lakomkin , Cornelius Weber , Stefan Wermter