English
Related papers

Related papers: The Behavior Gap: Evaluating Zero-shot LLM Agents …

200 papers

Large language models (LLMs) gained immense popularity due to their impressive capabilities in unstructured conversations. Empowering LLMs with advanced prompting strategies such as reasoning and acting (ReAct) (Yao et al., 2022) has shown…

Computation and Language · Computer Science 2025-03-18 Michelle Elizabeth , Morgan Veyret , Miguel Couceiro , Ondrej Dusek , Lina M. Rojas-Barahona

Large Language Models (LLMs) have been increasingly used in real-world settings, yet their strategic decision-making abilities remain largely unexplored. To fully benefit from the potential of LLMs, it's essential to understand their…

Artificial Intelligence · Computer Science 2024-10-16 Nathan Herr , Fernando Acero , Roberta Raileanu , María Pérez-Ortiz , Zhibin Li

Large Language Model (LLM) Agents have recently garnered increasing interest yet they are limited in their ability to learn from trial and error, a key element of intelligent behavior. In this work, we argue that the capacity to learn new…

Artificial Intelligence · Computer Science 2024-08-09 Haiteng Zhao , Chang Ma , Guoyin Wang , Jing Su , Lingpeng Kong , Jingjing Xu , Zhi-Hong Deng , Hongxia Yang

The emergence of instruction-tuned large language models (LLMs) has advanced the field of dialogue systems, enabling both realistic user simulations and robust multi-turn conversational agents. However, existing research often evaluates…

Computation and Language · Computer Science 2025-07-22 Chalamalasetti Kranti , Sherzod Hakimov , David Schlangen

Large Language Models (LLMs) are increasingly deployed as autonomous agents capable of actions with real-world impacts beyond text generation. While persona-induced biases in text generation are well documented, their effects on agent task…

Computation and Language · Computer Science 2026-02-16 Linbo Cao , Lihao Sun , Yang Yue

A successful negotiation requires a range of capabilities, including comprehension of the conversation context, Theory-of-Mind (ToM) skills to infer the partner's motives, strategic reasoning, and effective communication, making it…

Computation and Language · Computer Science 2024-10-03 Deuksin Kwon , Emily Weiss , Tara Kulshrestha , Kushal Chawla , Gale M. Lucas , Jonathan Gratch

Large Language Model (LLM) agents have been increasingly adopted as simulation tools to model humans in social science and role-playing applications. However, one fundamental question remains: can LLM agents really simulate human behavior?…

Artificial Intelligence · Computer Science 2024-11-04 Chengxing Xie , Canyu Chen , Feiran Jia , Ziyu Ye , Shiyang Lai , Kai Shu , Jindong Gu , Adel Bibi , Ziniu Hu , David Jurgens , James Evans , Philip Torr , Bernard Ghanem , Guohao Li

Recent advances in large language models (LLMs) have sparked growing interest in building fully autonomous agents. However, fully autonomous LLM-based agents still face significant challenges, including limited reliability due to…

Task-oriented dialogue systems (TODS) are continuing to rise in popularity as various industries find ways to effectively harness their capabilities, saving both time and money. However, even state-of-the-art TODS are not yet reaching their…

Computation and Language · Computer Science 2022-09-07 Ryan Fellows , Hisham Ihshaish , Steve Battle , Ciaran Haines , Peter Mayhew , J. Ignacio Deza

Large Language Models (LLMs) have become foundational to modern AI agent systems, enabling autonomous agents to reason and plan. In most existing systems, inter-agent communication relies primarily on natural language. While this design…

Artificial Intelligence · Computer Science 2025-06-04 Pengcheng Zhou , Yinglun Feng , Halimulati Julaiti , Zhongliang Yang

Solving complex or long-horizon problems often requires large language models (LLMs) to use external tools and operate over a significantly longer context window. New LLMs enable longer context windows and support tool calling capabilities.…

Machine Learning · Computer Science 2025-12-03 Tsimur Hadeliya , Mohammad Ali Jauhar , Nidhi Sakpal , Diogo Cruz

Large language models (LLMs) are increasingly used in social science simulations. While their performance on reasoning and optimization tasks has been extensively evaluated, less attention has been paid to their ability to simulate human…

Computational Engineering, Finance, and Science · Computer Science 2025-08-25 Yuanjun Feng , Vivek Choudhary , Yash Raj Shrestha

The ability of Large Language Models (LLMs) to extract context from natural language problem descriptions naturally raises questions about their suitability in autonomous decision-making settings. This paper studies the behaviour of these…

Artificial Intelligence · Computer Science 2025-07-22 Xiao Yang , Juxi Leitner , Michael Burke

Recent advances in task-oriented dialogue (TOD) systems, driven by large language models (LLMs) with extensive API and tool integration, have enabled conversational agents to coordinate interleaved goals, maintain long-horizon context, and…

Computation and Language · Computer Science 2026-02-02 Yifei Zhang , Hooshang Nayyeri , Rinat Khaziev , Emine Yilmaz , Gokhan Tur , Dilek Hakkani-Tür , Hari Thadakamalla

Large language models (LLMs) have demonstrated remarkable performance in zero-shot dialogue state tracking (DST), reducing the need for task-specific training. However, conventional DST benchmarks primarily focus on structured user-agent…

Computation and Language · Computer Science 2025-06-13 Sangmin Song , Juhwan Choi , JungMin Yun , YoungBin Kim

There is an growing interest in using Large Language Models (LLMs) in multi-agent systems to tackle interactive real-world tasks that require effective collaboration and assessing complex situations. Yet, we still have a limited…

Computation and Language · Computer Science 2024-06-11 Sahar Abdelnabi , Amr Gomaa , Sarath Sivaprasad , Lea Schönherr , Mario Fritz

Coding agents represent a new paradigm in automated software engineering, combining the reasoning capabilities of Large Language Models (LLMs) with tool-augmented interaction loops. However, coding agents still have severe limitations.…

Software Engineering · Computer Science 2026-04-06 Tural Mehtiyev , Wesley Assunção

Large Language Models (LLMs) generate text token-by-token in discrete time, yet real-world communication, from therapy sessions to business negotiations, critically depends on continuous time constraints. Current LLM architectures and…

Artificial Intelligence · Computer Science 2026-01-21 Neil K. R. Sehgal , Sharath Chandra Guntuku , Lyle Ungar

Task-oriented dialogue (TOD) systems are experiencing a revolution driven by Large Language Models (LLMs), yet the evaluation methodologies for these systems remain insufficient for their growing sophistication. While traditional automatic…

Computation and Language · Computer Science 2025-07-17 Emre Can Acikgoz , Carl Guo , Suvodip Dey , Akul Datta , Takyoung Kim , Gokhan Tur , Dilek Hakkani-Tür

Large Language Models (LLMs) have showcased remarkable capabilities in various Natural Language Processing tasks. For automatic open-domain dialogue evaluation in particular, LLMs have been seamlessly integrated into evaluation frameworks,…

Computation and Language · Computer Science 2024-07-08 John Mendonça , Alon Lavie , Isabel Trancoso