English
Related papers

Related papers: Improving Open-Domain Dialogue Evaluation with a C…

200 papers

While automatic response generation for building chatbot systems has drawn a lot of attention recently, there is limited understanding on when we need to consider the linguistic context of an input text in the generation process. The task…

Computation and Language · Computer Science 2016-11-04 Chaozhuo Li , Yu Wu , Wei Wu , Chen Xing , Zhoujun Li , Ming Zhou

In task-oriented conversational AI evaluation, unsupervised methods poorly correlate with human judgments, and supervised approaches lack generalization. Recent advances in large language models (LLMs) show robust zeroshot and few-shot…

Computation and Language · Computer Science 2024-06-26 Jinghan Jia , Abi Komma , Timothy Leffel , Xujun Peng , Ajay Nagesh , Tamer Soliman , Aram Galstyan , Anoop Kumar

Estimation of a model's confidence on its outputs is critical for Conversational AI systems based on large language models (LLMs), especially for reducing hallucination and preventing over-reliance. In this work, we provide an exhaustive…

Computation and Language · Computer Science 2024-09-24 Yi-Jyun Sun , Suvodip Dey , Dilek Hakkani-Tur , Gokhan Tur

To improve user engagement during conversations with dialogue systems, we must improve individual dialogue responses and dialogue impressions such as consistency, personality, and empathy throughout the entire dialogue. While such dialogue…

Computation and Language · Computer Science 2025-01-28 Kai Yoshida , Masahiro Mizukami , Seiya Kawano , Canasai Kruengkrai , Hiroaki Sugiyama , Koichiro Yoshino

We introduce an LSTM-based method for dynamically integrating several word-prediction experts to obtain a conditional language model which can be good simultaneously at several subtasks. We illustrate this general approach with an…

Artificial Intelligence · Computer Science 2016-05-06 Phong Le , Marc Dymetman , Jean-Michel Renders

This paper introduces a novel Self-supervised Fine-grained Dialogue Evaluation framework (SelF-Eval). The core idea is to model the correlation between turn quality and the entire dialogue quality. We first propose a novel automatic data…

Computation and Language · Computer Science 2022-09-19 Longxuan Ma , Ziyu Zhuang , Weinan Zhang , Mingda Li , Ting Liu

Although human evaluation remains the gold standard for open-domain dialogue evaluation, the growing popularity of automated evaluation using Large Language Models (LLMs) has also extended to dialogue. However, most frameworks leverage…

Computation and Language · Computer Science 2024-10-07 John Mendonça , Isabel Trancoso , Alon Lavie

Human feedback is crucial in the interactions between humans and Large Language Models (LLMs). However, existing research primarily focuses on benchmarking LLMs in single-turn dialogues. Even in benchmarks designed for multi-turn dialogues,…

Computation and Language · Computer Science 2025-02-18 Youquan Li , Miao Zheng , Fan Yang , Guosheng Dong , Bin Cui , Weipeng Chen , Zenan Zhou , Wentao Zhang

Dialogue structure discovery is essential in dialogue generation. Well-structured topic flow can leverage background information and predict future topics to help generate controllable and explainable responses. However, most previous work…

Computation and Language · Computer Science 2023-03-03 Congchi Yin , Piji Li , Zhaochun Ren

Large Language Models (LLMs) have showcased remarkable capabilities in various Natural Language Processing tasks. For automatic open-domain dialogue evaluation in particular, LLMs have been seamlessly integrated into evaluation frameworks,…

Computation and Language · Computer Science 2024-07-08 John Mendonça , Alon Lavie , Isabel Trancoso

Explainable recommender systems are designed to elucidate the explanation behind each recommendation, enabling users to comprehend the underlying logic. Previous works perform rating prediction and explanation generation in a multi-task…

Information Retrieval · Computer Science 2025-04-09 Shijie Liu , Ruixing Ding , Weihai Lu , Jun Wang , Mo Yu , Xiaoming Shi , Wei Zhang

Open-domain dialogue systems have started to engage in continuous conversations with humans. Those dialogue systems are required to be adjusted to the human interlocutor and evaluated in terms of their perspective. However, it is…

Computation and Language · Computer Science 2024-01-05 Yuma Tsuta , Naoki Yoshinaga , Shoetsu Sato , Masashi Toyoda

We propose a novel attention based hierarchical LSTM model to classify discourse act sequences in social media conversations, aimed at mining data from online discussion using textual meanings beyond sentence level. The very uniqueness of…

Computation and Language · Computer Science 2019-07-16 Subhabrata Dutta , Tanmoy Chakraborty , Dipankar Das

Large Language Models (LLMs) have achieved remarkable performance in objective tasks such as open-domain question answering and mathematical reasoning, which can often be solved through recalling learned factual knowledge or…

Computation and Language · Computer Science 2024-02-28 Xiaolong Wang , Yile Wang , Yuanchi Zhang , Fuwen Luo , Peng Li , Maosong Sun , Yang Liu

Large Language models are revolutionizing the conversational recommender systems through their impressive capabilities in instruction comprehension, reasoning, and human interaction. A core factor underlying effective recommendation…

Artificial Intelligence · Computer Science 2025-12-01 Mengfan Li , Xuanhua Shi , Yang Deng

Sequential recommendation systems aim to predict users' next likely interaction based on their history. However, these systems face data sparsity and cold-start problems. Utilizing data from other domains, known as multi-domain methods, is…

Information Retrieval · Computer Science 2025-02-20 Zuoli Tang , Zhaoxin Huan , Zihao Li , Xiaolu Zhang , Jun Hu , Chilin Fu , Jun Zhou , Lixin Zou , Chenliang Li

Pre-trained language models (LLMs) such as GPT-3 can carry fluent, multi-turn conversations out-of-the-box, making them attractive materials for chatbot design. Further, designers can improve LLM chatbot utterances by prepending textual…

Human-Computer Interaction · Computer Science 2023-02-08 J. D. Zamfirescu-Pereira , Bjoern Hartmann , Qian Yang

Causality is vital for understanding true cause-and-effect relationships between variables within predictive models, rather than relying on mere correlations, making it highly relevant in the field of Explainable AI. In an automated…

Machine Learning · Computer Science 2024-08-28 Arturo Fredes , Jordi Vitria

Encoder-decoder based neural architectures serve as the basis of state-of-the-art approaches in end-to-end open domain dialog systems. Since most of such systems are trained with a maximum likelihood~(MLE) objective they suffer from issues…

The construction of open-domain dialogue systems requires high-quality dialogue datasets. The dialogue data admits a wide variety of responses for a given dialogue history, especially responses with different semantics. However, collecting…

Computation and Language · Computer Science 2022-11-01 Jiao Ou , Jinchao Zhang , Yang Feng , Jie Zhou