English
Related papers

Related papers: How Different AI Chatbots Behave? Benchmarking Lar…

200 papers

Large language models (LLMs) increasingly mediate economic and organisational processes, from automated customer support and recruitment to investment advice and policy analysis. These systems are often assumed to embody rational decision…

General Economics · Economics 2025-11-18 Luca Corazzini , Elisa Deriu , Marco Guerzoni

As large language models (LLMs) advance across diverse tasks, the need for comprehensive evaluation beyond single metrics becomes increasingly important. To fully assess LLM intelligence, it is crucial to examine their interactive dynamics…

Computation and Language · Computer Science 2025-09-23 Junhao Chen , Jingbo Sun , Xiang Li , Haidong Xin , Yuhao Xue , Yibin Xu , Hao Zhao

The integration of Large Language Models (LLMs) and chatbots introduces new challenges and opportunities for decision-making in software testing. Decision-making relies on a variety of information, including code, requirements…

Software Engineering · Computer Science 2024-06-18 Francisco Gomes de Oliveira Neto

Given the advancements in conversational artificial intelligence, the evaluation and assessment of Large Language Models (LLMs) play a crucial role in ensuring optimal performance across various conversational tasks. In this paper, we…

Large language models (LLMs) are increasingly proposed as agents in strategic decision environments, yet their behavior in structured geopolitical simulations remains under-researched. We evaluate six popular state-of-the-art LLMs alongside…

Computation and Language · Computer Science 2026-03-03 Veronika Solopova , Viktoria Skorik , Maksym Tereshchenko , Alina Haidun , Ostap Vykhopen

Large Language Models (LLMs) are increasingly being used in learning environments to support teaching-be it as learning companions or as tutors. With our contribution, we aim to discuss the implications of the anthropomorphization of LLMs…

Computation and Language · Computer Science 2024-08-09 Kristina Schaaff , Marc-André Heidelmann

As Large Language Models (LLMs) are increasingly deployed in social and strategic scenarios, it becomes critical to understand where and why their behavior diverges from that of humans. While behavioral game theory (BGT) provides a…

Artificial Intelligence · Computer Science 2026-02-12 Caroline Wang , Daniel Kasenberg , Kim Stachenfeld , Pablo Samuel Castro

Large Language Models (LLMs) have shown remarkable promise in communicating with humans. Their potential use as artificial partners with humans in sociological experiments involving conversation is an exciting prospect. But how viable is…

Artificial Intelligence · Computer Science 2025-02-04 James Flamino , Mohammed Shahid Modi , Boleslaw K. Szymanski , Brendan Cross , Colton Mikolajczyk

Each day, individuals set behavioral goals such as eating healthier, exercising regularly, or increasing productivity. While psychological frameworks (i.e., goal setting and implementation intentions) can be helpful, they often need…

Human-Computer Interaction · Computer Science 2026-02-10 Michel Schimpf , Sebastian Maier , Anton Wyrowski , Lara Christoforakos , Stefan Feuerriegel , Thomas Bohné

As LLMs increasingly act as autonomous agents in interactive and multi-agent settings, understanding their strategic behavior is critical for safety, coordination, and AI-driven social and economic systems. We investigate how payoff…

We explore the potential of Large Language Models (LLMs) to replicate human behavior in economic market experiments. Compared to previous studies, we focus on dynamic feedback between LLM agents: the decisions of each LLM impact the market…

General Economics · Economics 2025-05-13 R. Maria del Rio-Chanona , Marco Pangallo , Cars Hommes

As Large Language Models (LLMs) are integrated into critical real-world applications, their strategic and logical reasoning abilities are increasingly crucial. This paper evaluates LLMs' reasoning abilities in competitive environments…

Computation and Language · Computer Science 2024-06-11 Jinhao Duan , Renming Zhang , James Diffenderfer , Bhavya Kailkhura , Lichao Sun , Elias Stengel-Eskin , Mohit Bansal , Tianlong Chen , Kaidi Xu

With the rapid advancement of Large Language Models (LLMs), recent studies have drawn attention to their potential for handling not only simple question-answer tasks but also more complex conversational abilities and performing human-like…

Artificial Intelligence · Computer Science 2025-11-25 Mingyu Jeon , Jaeyoung Suh , Suwan Cho , Dohyeon Kim

Large language models (LLMs) exhibit expert-level performance in tasks across a wide range of different domains. Ethical issues raised by LLMs and the need to align future versions makes it important to know how state of the art models…

Artificial Intelligence · Computer Science 2024-05-24 Guilherme F. C. F. Almeida , José Luiz Nunes , Neele Engelmann , Alex Wiegmann , Marcelo de Araújo

Large language models (LLMs) show increasingly advanced emergent capabilities and are being incorporated across various societal domains. Understanding their behavior and reasoning abilities therefore holds significant importance. We argue…

Computation and Language · Computer Science 2024-08-09 Thilo Hagendorff , Ishita Dasgupta , Marcel Binz , Stephanie C. Y. Chan , Andrew Lampinen , Jane X. Wang , Zeynep Akata , Eric Schulz

Large language models (LLMs) excel at solving problems with clear and complete statements, but often struggle with nuanced environments or interactive tasks which are common in most real-world scenarios. This highlights the critical need…

Large Language Models (LLMs) show significant potential in economic and strategic interactions, where communication via natural language is often prevalent. This raises key questions: Do LLMs behave rationally? How do they perform compared…

Computation and Language · Computer Science 2026-03-03 Eilam Shapira , Omer Madmon , Itamar Reinman , Samuel Joseph Amouyal , Roi Reichart , Moshe Tennenholtz

When encountering increasingly frequent performance improvements or cost reductions from a new large language model (LLM), developers of applications leveraging LLMs must decide whether to take advantage of these improvements or stay with…

Computation and Language · Computer Science 2025-02-20 Rubing Li , João Sedoc , Arun Sundararajan

As Large Language Models (LLMs) increasingly operate as autonomous decision-makers in interactive and multi-agent systems and human societies, understanding their strategic behaviour has profound implications for safety, coordination, and…

The emergence of Large Language Models (LLMs), has opened exciting possibilities for constructing computational simulations designed to replicate human behavior accurately. Current research suggests that LLM-based agents become increasingly…

Computation and Language · Computer Science 2024-12-18 Amir Taubenfeld , Yaniv Dover , Roi Reichart , Ariel Goldstein