English
Related papers

Related papers: FANToM: A Benchmark for Stress-testing Machine The…

200 papers

Theory of Mind (ToM) is the ability to reason about one's own and others' mental states. ToM plays a critical role in the development of intelligence, language understanding, and cognitive processes. While previous work has primarily…

Computation and Language · Computer Science 2023-10-26 Yinghui He , Yufan Wu , Yilin Jia , Rada Mihalcea , Yulong Chen , Naihao Deng

While recent studies explore Large Language Models' (LLMs) performance on Theory of Mind (ToM) reasoning tasks, research on ToM abilities that require more nuanced social context is limited, such as white lies. We introduce TactfulToM, a…

Computation and Language · Computer Science 2025-09-26 Yiwei Liu , Emma Jane Pretty , Jiahao Huang , Saku Sugawara

Our paper argues that the majority of theory of mind benchmarks are broken because of their inability to directly test how large language models (LLMs) adapt to new partners. This problem stems from the fact that theory of mind benchmarks…

Artificial Intelligence · Computer Science 2025-06-13 Matthew Riemer , Zahra Ashktorab , Djallel Bouneffouf , Payel Das , Miao Liu , Justin D. Weisz , Murray Campbell

Human interactions are deeply rooted in the interplay of thoughts, beliefs, and desires made possible by Theory of Mind (ToM): our cognitive ability to understand the mental states of ourselves and others. Although ToM may come naturally to…

Artificial Intelligence · Computer Science 2023-11-20 Alex Wilf , Sihyun Shawn Lee , Paul Pu Liang , Louis-Philippe Morency

Theory of Mind (ToM), the ability to track others epistemic state, makes humans efficient collaborators. AI agents need the same capacity in multi agent settings, yet existing benchmarks mostly test literal ToM by asking direct belief…

Artificial Intelligence · Computer Science 2026-05-19 Gurusha Juneja , Dylan Lu , Saaket Agashe , Parth Diwane , Edward Gunn , Jayanth Srinivasa , Gaowen Liu , William Yang Wang , Yali Du , Xin Eric Wang

Evaluating the theory of mind (ToM) capabilities of language models (LMs) has recently received a great deal of attention. However, many existing benchmarks rely on synthetic data, which risks misaligning the resulting experiments with…

Computation and Language · Computer Science 2024-06-11 Adil Soubki , John Murzaku , Arash Yousefi Jordehi , Peter Zeng , Magdalena Markowska , Seyed Abolghasem Mirroshandel , Owen Rambow

Theory-of-Mind (ToM), the ability to infer others' perceptions and mental states, is fundamental to human interaction but remains challenging for Large Language Models (LLMs). While existing ToM reasoning methods show promise with reasoning…

Computation and Language · Computer Science 2025-06-03 Hainiu Xu , Siya Qi , Jiazheng Li , Yuxiang Zhou , Jinhua Du , Caroline Catmur , Yulan He

Theory of Mind (ToM) is the ability to understand and reflect on the mental states of others. Although this capability is crucial for human interaction, testing on Large Language Models (LLMs) reveals that they possess only a rudimentary…

Computation and Language · Computer Science 2025-01-17 Sneheel Sarangi , Maha Elgarf , Hanan Salam

As large language models (LLMs) continue to advance, there is increasing interest in their ability to infer human mental states and demonstrate a human-like Theory of Mind (ToM). Most existing ToM evaluations, however, are centered on…

Computer Vision and Pattern Recognition · Computer Science 2026-03-26 Siqi Liu , Xinyang Li , Bochao Zou , Junbao Zhuo , Huimin Ma , Jiansheng Chen

Theory of Mind (ToM) refers to an agent's ability to model the internal states of others. Contributing to the debate whether large language models (LLMs) exhibit genuine ToM capabilities, our study investigates their ToM robustness using…

Computation and Language · Computer Science 2026-02-26 Christian Nickel , Laura Schrewe , Florian Mai , Lucie Flek

Datasets used for emotion recognition tasks typically contain overt cues that can be used in predicting the emotions expressed in a text. However, one challenge is that texts sometimes contain covert contextual cues that are rich in…

Computation and Language · Computer Science 2025-06-03 Gerard Christopher Yeo , Kokil Jaidka

The question of whether large language models (LLMs) possess Theory of Mind (ToM) -- often defined as the ability to reason about others' mental states -- has sparked significant scientific and public interest. However, the evidence as to…

Artificial Intelligence · Computer Science 2025-03-03 Jennifer Hu , Felix Sosa , Tomer Ullman

While humans naturally develop theory of mind (ToM), the capability to understand other people's mental states and beliefs, state-of-the-art large language models (LLMs) underperform on simple ToM benchmarks. We posit that we can extend our…

Computation and Language · Computer Science 2024-11-08 Chani Jung , Dongkwan Kim , Jiho Jin , Jiseon Kim , Yeon Seonwoo , Yejin Choi , Alice Oh , Hyunwoo Kim

In recent years, evaluating the Theory of Mind (ToM) capabilities of large language models (LLMs) has received significant attention within the research community. As the field rapidly evolves, navigating the diverse approaches and…

Computation and Language · Computer Science 2025-02-14 Karahan Sarıtaş , Kıvanç Tezören , Yavuz Durmazkeser

Large Language Models (LLMs) have generated considerable interest and debate regarding their potential emergence of Theory of Mind (ToM). Several recent inquiries reveal a lack of robust ToM in these models and pose a pressing demand to…

Computation and Language · Computer Science 2024-12-30 Ziqiao Ma , Jacob Sansom , Run Peng , Joyce Chai

As large language models evolve, there is growing anticipation that they will emulate human-like Theory of Mind (ToM) to assist with routine tasks. However, existing methods for evaluating machine ToM focus primarily on unimodal models and…

Artificial Intelligence · Computer Science 2025-06-18 Xinyang Li , Siqi Liu , Bochao Zou , Jiansheng Chen , Huimin Ma

Understanding Theory of Mind is essential for building socially intelligent multimodal agents capable of perceiving and interpreting human behavior. We introduce MoMentS (Multimodal Mental States), a comprehensive benchmark designed to…

Theory of Mind (ToM)-an understanding of the mental states of others-is a key aspect of human social intelligence, yet, chatbots and LLM-based social agents do not typically integrate it. In this work, we demonstrate that LLMs that…

Computation and Language · Computer Science 2026-04-14 EunJeong Hwang , Yuwei Yin , Giuseppe Carenini , Peter West , Vered Shwartz

Theory of Mind (ToM), the ability to understand people's minds based on their behavior, is key to developing socially intelligent agents. Current approaches to ToM reasoning either rely on prompting Large Language Models (LLMs), which are…

Artificial Intelligence · Computer Science 2026-01-15 Zhining Zhang , Chuanyang Jin , Mung Yao Jia , Shunchi Zhang , Tianmin Shu

Do Large Language Models (LLMs) possess a Theory of Mind (ToM)? Research into this question has focused on evaluating LLMs against benchmarks and found success across a range of social tasks. However, these evaluations do not test for the…

Artificial Intelligence · Computer Science 2026-02-27 John Muchovej , Amanda Royka , Shane Lee , Julian Jara-Ettinger