English
Related papers

Related papers: User Simulation with Large Language Models for Eva…

200 papers

Task-oriented dialogue (TOD) systems enable users to achieve their goals through natural language interactions. Traditionally, these systems have relied on turn-level manually annotated metadata, such as dialogue states and policy…

Computation and Language · Computer Science 2024-11-05 Adib Mosharrof , A. B. Siddique

In the realm of dialogue systems, user simulation techniques have emerged as a game-changer, redefining the evaluation and enhancement of task-oriented dialogue (TOD) systems. These methods are crucial for replicating real user…

User simulation has long played a vital role in computer science due to its potential to support a wide range of applications. Language, as the primary medium of human communication, forms the foundation of social interaction and behavior.…

Recent advances in task-oriented dialogue (TOD) systems, driven by large language models (LLMs) with extensive API and tool integration, have enabled conversational agents to coordinate interleaved goals, maintain long-horizon context, and…

Computation and Language · Computer Science 2026-02-02 Yifei Zhang , Hooshang Nayyeri , Rinat Khaziev , Emine Yilmaz , Gokhan Tur , Dilek Hakkani-Tür , Hari Thadakamalla

Large language models (LLMs) have been used for diverse tasks in natural language processing (NLP), yet remain under-explored for task-oriented dialogue systems (TODS), especially for end-to-end TODS. We present InstructTODS, a novel…

Computation and Language · Computer Science 2023-10-16 Willy Chung , Samuel Cahyawijaya , Bryan Wilie , Holy Lovenia , Pascale Fung

Task oriented dialogue systems (TOD) complete particular tasks based on user preferences across natural language interactions. Considering the impressive performance of large language models (LLMs) in natural language processing (NLP)…

Computation and Language · Computer Science 2025-10-08 Arezoo Saedi , Afsaneh Fatemi , Mohammad Ali Nematbakhsh , Sophie Rosset , Anne Vilnat

Proactive task-oriented dialogue (TOD), such as outbound sales, demands a persuasive agent that actively probes the user's concerns and steers the conversation toward acceptance within a bounded number of turns. Yet post-trained LLMs are…

Artificial Intelligence · Computer Science 2026-05-22 Hongbin Zhang , Ning Gao , Yuqin Dai , Ruiyuan Wu , Jinpeng Wang , Rena Wei Gao , Bingdong Tan , Shuzheng Gao , Zongjie Li , Chaozheng Wang

This paper presents a novel application of large language models in user simulation for task-oriented dialog systems, specifically focusing on an in-context learning approach. By harnessing the power of these models, the proposed approach…

Computation and Language · Computer Science 2023-06-02 Silvia Terragni , Modestas Filipavicius , Nghia Khau , Bruna Guedes , André Manso , Roland Mathis

Task-oriented dialogue (TOD) systems facilitate users in executing various activities via multi-turn dialogues, but Large Language Models (LLMs) often struggle to comprehend these intricate contexts. In this study, we propose a novel…

Computation and Language · Computer Science 2023-09-25 Haoyu Gao , Ting-En Lin , Hangyu Li , Min Yang , Yuchuan Wu , Wentao Ma , Yongbin Li

Robust task-oriented spoken dialogue agents require exposure to the full diversity of how people interact through speech. Building spoken user simulators that address this requires large-scale spoken task-oriented dialogue (TOD) data…

Computation and Language · Computer Science 2026-03-18 Jonggeun Lee , Junseong Pyo , Jeongmin Park , Yohan Jo

This paper explores SynTOD, a new synthetic data generation approach for developing end-to-end Task-Oriented Dialogue (TOD) Systems capable of handling complex tasks such as intent classification, slot filling, conversational…

Computation and Language · Computer Science 2024-04-24 Chris Samarinas , Pracha Promthaw , Atharva Nijasure , Hansi Zeng , Julian Killingback , Hamed Zamani

In recent research on dialogue systems and corpora, there has been a significant focus on two distinct categories: task-oriented (TOD) and open-domain (chit-chat) dialogues. TOD systems aim to satisfy specific user goals, such as finding a…

Computation and Language · Computer Science 2023-08-29 Wen-Yu Chang , Yun-Nung Chen

We present DIALIGHT, a toolkit for developing and evaluating multilingual Task-Oriented Dialogue (ToD) systems which facilitates systematic evaluations and comparisons between ToD systems using fine-tuning of Pretrained Language Models…

Computation and Language · Computer Science 2024-01-05 Songbo Hu , Xiaobin Wang , Zhangdie Yuan , Anna Korhonen , Ivan Vulić

This survey provides a comprehensive review of research on multi-turn dialogue systems, with a particular focus on multi-turn dialogue systems based on large language models (LLMs). This paper aims to (a) give a summary of existing LLMs and…

Computation and Language · Computer Science 2025-08-18 Zihao Yi , Jiarui Ouyang , Zhe Xu , Yuwen Liu , Tianhao Liao , Haohao Luo , Ying Shen

Evaluation is crucial in the development process of task-oriented dialogue systems. As an evaluation method, user simulation allows us to tackle issues such as scalability and cost-efficiency, making it a viable choice for large-scale…

Information Retrieval · Computer Science 2021-05-11 Weiwei Sun , Shuo Zhang , Krisztian Balog , Zhaochun Ren , Pengjie Ren , Zhumin Chen , Maarten de Rijke

Task-oriented dialogue (TOD) systems aim to efficiently handle task-oriented conversations, including information collection. How to utilize TOD accurately, efficiently and effectively for information collection has always been a critical…

Artificial Intelligence · Computer Science 2024-10-15 Ming Zhang , Caishuang Huang , Yilong Wu , Shichun Liu , Huiyuan Zheng , Yurui Dong , Yujiong Shen , Shihan Dou , Jun Zhao , Junjie Ye , Qi Zhang , Tao Gui , Xuanjing Huang

Dialogue systems (DS), including the task-oriented dialogue system (TOD) and the open-domain dialogue system (ODD), have always been a fundamental task in natural language processing (NLP), allowing various applications in practice. Owing…

Computation and Language · Computer Science 2025-07-22 Hongru Wang , Lingzhi Wang , Yiming Du , Liang Chen , Jingyan Zhou , Yufei Wang , Kam-Fai Wong

Task-Oriented Dialogue (TOD) systems are designed to carry out specific tasks by tracking dialogue states and generating appropriate responses to help users achieve defined goals. Recently, end-to-end dialogue models pre-trained based on…

Computation and Language · Computer Science 2023-06-01 Namo Bang , Jeehyun Lee , Myoung-Wan Koo

Detecting user frustration in modern-day task-oriented dialog (TOD) systems is imperative for maintaining overall user satisfaction, engagement, and retention. However, most recent research is focused on sentiment and emotion detection in…

Embodied agents designed to assist users with tasks must engage in natural language interactions, interpret instructions, execute actions, and communicate effectively to resolve issues. However, collecting large-scale, diverse datasets of…

Computation and Language · Computer Science 2024-11-01 Daniel Philipov , Vardhan Dongre , Gokhan Tur , Dilek Hakkani-Tür