English
Related papers

Related papers: User Simulation with Large Language Models for Eva…

200 papers

Gestures perform a variety of communicative functions that powerfully influence human face-to-face interaction. How this communicative function is achieved varies greatly between individuals and depends on the role of the speaker and the…

Human-Computer Interaction · Computer Science 2023-10-24 Laura B. Hensel , Nutchanon Yongsatianchot , Parisa Torshizi , Elena Minucci , Stacy Marsella

Task-Oriented Dialogue (TOD) systems have become crucial components in interactive artificial intelligence applications. While recent advances have capitalized on pre-trained language models (PLMs), they exhibit limitations regarding…

Computation and Language · Computer Science 2023-12-11 Sungryull Sohn , Yiwei Lyu , Anthony Liu , Lajanugen Logeswaran , Dong-Ki Kim , Dongsub Shim , Honglak Lee

Existing benchmarks that assess Language Models (LMs) as Language Agents (LAs) for tool use primarily focus on stateless, single-turn interactions or partial evaluations, such as tool selection in a single turn, overlooking the inherent…

Computation and Language · Computer Science 2025-05-20 Hongru Wang , Wenyu Huang , Yufei Wang , Yuanhao Xi , Jianqiao Lu , Huan Zhang , Nan Hu , Zeming Liu , Jeff Z. Pan , Kam-Fai Wong

Dialogue State Tracking (DST) is designed to monitor the evolving dialogue state in the conversations and plays a pivotal role in developing task-oriented dialogue systems. However, obtaining the annotated data for the DST task is usually a…

Computation and Language · Computer Science 2024-05-24 Cheng Niu , Xingguang Wang , Xuxin Cheng , Juntong Song , Tong Zhang

Using Large Language Models (LLMs) to simulate user opinions has received growing attention. Yet LLMs, especially trained with reinforcement learning from human feedback (RLHF), are known to exhibit biases toward dominant viewpoints,…

Computation and Language · Computer Science 2025-12-09 Ziyun Yu , Yiru Zhou , Chen Zhao , Hongyi Wen

Task-Oriented Dialogue (TOD) systems assist users in completing tasks through natural language interactions, often relying on a single-layered workflow structure for slot-filling in public tasks, such as hotel bookings. However, in…

Computation and Language · Computer Science 2025-09-09 Lingbo Mo , Shun Jiang , Akash Maharaj , Bernard Hishamunda , Yunyao Li

With growing capabilities of large language models (LLMs) comes growing affordances for human-like and context-aware conversational partners. On from this, some recent work has investigated the use of LLMs to simulate multiple…

Human-Computer Interaction · Computer Science 2023-12-29 Samuel Rhys Cox

Large Language Models (LLMs) with API-calling capabilities enabled building effective Language Agents (LA), while also revolutionizing the conventional task-oriented dialogue (TOD) paradigm. However, current approaches face a critical…

Recent Multimodal Large Language Models (MLLMs) have typically focused on integrating visual and textual modalities, with less emphasis placed on the role of speech in enhancing interaction. However, speech plays a crucial role in…

Computer Vision and Pattern Recognition · Computer Science 2025-10-27 Chaoyou Fu , Haojia Lin , Xiong Wang , Yi-Fan Zhang , Yunhang Shen , Xiaoyu Liu , Haoyu Cao , Zuwei Long , Heting Gao , Ke Li , Long Ma , Xiawu Zheng , Rongrong Ji , Xing Sun , Caifeng Shan , Ran He

We present BotSIM, a data-efficient end-to-end Bot SIMulation toolkit for commercial text-based task-oriented dialog (TOD) systems. BotSIM consists of three major components: 1) a Generator that can infer semantic-level dialog acts and…

Computation and Language · Computer Science 2022-12-01 Guangsen Wang , Samson Tan , Shafiq Joty , Gang Wu , Jimmy Au , Steven Hoi

Task-oriented dialogue (TOD) systems have been widely used by mobile phone intelligent assistants to accomplish tasks such as calendar scheduling or hotel reservation. Current TOD systems usually focus on multi-turn text/speech interaction,…

Computation and Language · Computer Science 2024-03-04 Liangtai Sun , Xingyu Chen , Lu Chen , Tianle Dai , Zichen Zhu , Kai Yu

The tendency of users to anthropomorphise large language models (LLMs) is of growing interest to AI developers, researchers, and policy-makers. Here, we present a novel method for empirically evaluating anthropomorphic LLM behaviours in…

Large Language Model (LLM) agents have developed rapidly in recent years to solve complex real-world problems using external tools. However, the scarcity of high-quality trajectories still hinders the development of stronger LLM agents.…

Artificial Intelligence · Computer Science 2025-12-08 Chen Yang , Ran Le , Yun Xing , Zhenwei An , Zongchao Chen , Wayne Xin Zhao , Yang Song , Tao Zhang

Personalization in social robotics is critical for fostering effective human-robot interactions, yet systems often face the cold start problem, where initial user preferences or characteristics are unavailable. This paper proposes a novel…

Human-Computer Interaction · Computer Science 2025-04-03 Hamed Rahimi , Jeanne Cattoni , Meriem Beghili , Mouad Abrini , Mahdi Khoramshahi , Maribel Pino , Mohamed Chetouani

Large Language Models (LLMs) have shown remarkable promise in communicating with humans. Their potential use as artificial partners with humans in sociological experiments involving conversation is an exciting prospect. But how viable is…

Artificial Intelligence · Computer Science 2025-02-04 James Flamino , Mohammed Shahid Modi , Boleslaw K. Szymanski , Brendan Cross , Colton Mikolajczyk

Conversational Recommender System (CRS) interacts with users through natural language to understand their preferences and provide personalized recommendations in real-time. CRS has demonstrated significant potential, prompting researchers…

Artificial Intelligence · Computer Science 2024-03-26 Lixi Zhu , Xiaowen Huang , Jitao Sang

Synthetic users are cost-effective proxies for real users in the evaluation of conversational recommender systems. Large language models show promise in simulating human-like behavior, raising the question of their ability to represent a…

Computation and Language · Computer Science 2024-03-27 Se-eun Yoon , Zhankui He , Jessica Maria Echterhoff , Julian McAuley

Spoken Dialogue Models (SDMs) have recently attracted significant attention for their ability to generate voice responses directly to users' spoken queries. Despite their increasing popularity, there exists a gap in research focused on…

Computation and Language · Computer Science 2025-10-07 Chengqian Ma , Wei Tao , Yiwen Guo

Collecting human-chatbot dialogues typically demands substantial manual effort and is time-consuming, which limits and poses challenges for research on conversational AI. In this work, we propose DialogueForge - a framework for generating…

Computation and Language · Computer Science 2025-07-22 Ruizhe Zhu , Hao Zhu , Yaxuan Li , Syang Zhou , Shijing Cai , Malgorzata Lazuka , Elliott Ash

Understanding how humans collaborate and communicate in teams is essential for improving human-agent teaming and AI-assisted decision-making. However, relying solely on data from large-scale user studies is impractical due to logistical,…