English
Related papers

Related papers: Lost in Simulation: LLM-Simulated Users are Unreli…

200 papers

Large language model (LLM)-based agents are increasingly deployed in e-commerce shopping. To perform thorough, user-tailored product searches, agents should interpret personal preferences, engage in multi-turn dialogues, and ultimately…

As AI systems advance in capabilities, measuring their safety and alignment to human values is becoming paramount. A fast-growing field of AI research is devoted to developing such assessments. However, most current advances therein may be…

Computers and Society · Computer Science 2026-03-17 Max Hellrigel-Holderbaum , Edward James Young

Large language models (LLMs) are increasingly used in social science simulations. While their performance on reasoning and optimization tasks has been extensively evaluated, less attention has been paid to their ability to simulate human…

Computational Engineering, Finance, and Science · Computer Science 2025-08-25 Yuanjun Feng , Vivek Choudhary , Yash Raj Shrestha

As Large Language Model (LLM) agents become more widespread, associated misalignment risks increase. While prior research has studied agents' ability to produce harmful outputs or follow malicious instructions, it remains unclear how likely…

As large language models (LLMs) are increasingly deployed as agents, their integration into interactive environments and tool use introduce new safety challenges beyond those associated with the models themselves. However, the absence of…

Computation and Language · Computer Science 2025-05-21 Zhexin Zhang , Shiyao Cui , Yida Lu , Jingzhuo Zhou , Junxiao Yang , Hongning Wang , Minlie Huang

Large language models (LLMs) have shown strong performance on standardized social science instruments, but their value for product discovery remains unclear. We investigate whether interview-informed generative agents can simulate user…

Human-Computer Interaction · Computer Science 2026-04-01 Zichao Wang , Alexa Siu

Large language models (LLMs) remain unreliable for global enterprise applications due to substantial performance gaps between high-resource and mid/low-resource languages, driven by English-centric pretraining and internal reasoning biases.…

Computation and Language · Computer Science 2025-10-28 Amit Agarwal , Hansa Meghwani , Hitesh Laxmichand Patel , Tao Sheng , Sujith Ravi , Dan Roth

Large language models (LLMs) are increasingly deployed as customer-facing agents, yet evaluating their reliability remains challenging due to stochastic, multi-turn interactions. Current evaluation protocols rely on linear Monte Carlo…

Artificial Intelligence · Computer Science 2026-04-24 Itay Nakash , George Kour , Ateret Anaby-Tavor

Large Language Models (LLMs) are increasingly used to simulate how specific users respond to a given context, enabling more user-centric applications that rely on user feedback. However, existing user simulators mostly imitate surface-level…

Computation and Language · Computer Science 2026-03-05 Shirley Wu , Evelyn Choi , Arpandeep Khatua , Zhanghan Wang , Joy He-Yueya , Tharindu Cyril Weerasooriya , Wei Wei , Diyi Yang , Jure Leskovec , James Zou

Evaluating recommender systems remains challenging due to the gap between offline metrics and real user behavior, as well as the scarcity of interaction data. Recent work explores large language model (LLM) agents as synthetic users, yet…

Information Retrieval · Computer Science 2026-01-06 Nicolas Bougie , Gian Maria Marconi , Tony Yip , Narimasa Watanabe

Large Language Model (LLM) agents are increasingly deployed in settings where they interact with a wide variety of people, including users who are unclear, impatient, or reluctant to share information. However, collecting real interaction…

Artificial Intelligence · Computer Science 2026-05-14 Harshita Chopra , Kshitish Ghate , Aylin Caliskan , Tadayoshi Kohno , Chirag Shah , Natasha Jaques

Using persona-conditioned LLMs as synthetic survey respondents has become a common practice in computational social science and agent-based simulations. Yet, it remains unclear whether multi-attribute persona prompting improves LLM…

Computers and Society · Computer Science 2026-02-24 Erika Elizabeth Taday Morocho , Lorenzo Cima , Tiziano Fagni , Marco Avvenuti , Stefano Cresci

Large language models (LLMs) demonstrate strong potential as agents for tool invocation due to their advanced comprehension and planning capabilities. Users increasingly rely on LLM-based agents to solve complex missions through iterative…

Artificial Intelligence · Computer Science 2025-04-17 Peijie Yu , Yifan Yang , Jinjian Li , Zelong Zhang , Haorui Wang , Xiao Feng , Feng Zhang

Simulating consumer decision-making is vital for designing and evaluating marketing strategies before costly real-world deployment. However, post-event analyses and rule-based agent-based models (ABMs) struggle to capture the complexity of…

Artificial Intelligence · Computer Science 2025-10-22 Man-Lin Chu , Lucian Terhorst , Kadin Reed , Tom Ni , Weiwei Chen , Rongyu Lin

Current Large Language Models (LLMs) are gradually exploited in practically valuable agentic workflows such as Deep Research, E-commerce recommendation, and job recruitment. In these applications, LLMs need to select some optimal solutions…

Computers and Society · Computer Science 2026-03-23 Zichen Tang , Zirui Zhang , Qian Wang , Zhenheng Tang , Bo Li , Xiaowen Chu

As AI agents increasingly act on behalf of human stakeholders in economic settings, understanding their behavior in complex market environments becomes critical. This article examines how Large Language Models coordinate on markets that are…

General Economics · Economics 2026-03-11 Alexander Erlei , Lukas Meub

Large language models (LLMs) can generate persuasive narratives at scale, raising concerns about their potential use in disinformation campaigns. Assessing this risk ultimately requires understanding how readers receive such content. In…

Artificial Intelligence · Computer Science 2026-04-09 Zonghuan Xu , Xiang Zheng , Yutao Wu , Xingjun Ma

Large Language Models (LLMs)-based agents have made impressive progress in reasoning and tool use, enabling them to solve complex tasks. However, their ability to proactively collaborate with users, especially when goals are vague,…

Recently, there has been an emergence of employing LLM-powered agents as believable human proxies, based on their remarkable decision-making capability. However, existing studies mainly focus on simulating human dialogue. Human non-verbal…

Information Retrieval · Computer Science 2023-10-16 Junjie Zhang , Yupeng Hou , Ruobing Xie , Wenqi Sun , Julian McAuley , Wayne Xin Zhao , Leyu Lin , Ji-Rong Wen

Recent advances in large language models (LLMs) have facilitated the widespread deployment of LLMs as interactive agents capable of reasoning, planning, and tool use. Despite strong performance on existing benchmarks, such agents often…

Artificial Intelligence · Computer Science 2026-05-27 Yuxin Chen , Xiaodong Cai , Junfeng Fang , Zhuowen Han , Yu Wang , Yaorui Shi , Yi Zhang , Qi Gu , Xunliang Cai , Xiang Wang , An Zhang , Tat-Seng Chua