When Large Language Models are More PersuasiveThan Incentivized Humans, and Why
Abstract
Large Language Models (LLMs) have been shown to be highly persuasive, but when and why they outperform humans is still an open question. We compare the persuasiveness of two LLMs (Claude 3.5 Sonnet and DeepSeek v3) against humans who had incentives to persuade, using an interactive, real-time conversational setting. We demonstrate that LLMs persuasive superiority is context-dependent: it depends on whether the persuasion attempt is truthful (towards the right answer) or deceptive (towards the wrong answer) and on the LLM model, and wanes over repeated interactions (unlike human persuasiveness). In our first large-scale experiment, humans vs LLMs (Claude 3.5 Sonnet) interacted with other humans who were completing an online quiz for a reward, attempting to persuade them toward a given (either correct or incorrect) answer. Claude was more persuasive than incentivized human persuaders both in truthful and deceptive contexts and it significantly increased accuracy if persuasion was truthful, but decreased it if persuasion was deceptive. In a follow-up experiment with Deepseek v3, we replicated the findings about accuracy but found greater LLM persuasiveness only if the persuasion was deceptive. Linguistic analyses of the persuaders texts suggest that these effects may be due to LLMs expressing higher conviction than humans.
Cite
@article{arxiv.2505.09662,
title = {When Large Language Models are More PersuasiveThan Incentivized Humans, and Why},
author = {Philipp Schoenegger and Francesco Salvi and Jiacheng Liu and Xiaoli Nan and Ramit Debnath and Barbara Fasolo and Evelina Leivada and Gabriel Recchia and Fritz Günther and Ali Zarifhonarvar and Joe Kwon and Zahoor Ul Islam and Marco Dehnert and Daryl Y. H. Lee and Madeline G. Reinecke and David G. Kamper and Mert Kobaş and Adam Sandford and Jonas Kgomo and Luke Hewitt and Shreya Kapoor and Kerem Oktar and Eyup Engin Kucuk and Bo Feng and Cameron R. Jones and Izzy Gainsburg and Sebastian Olschewski and Nora Heinzelmann and Francisco Cruz and Ben M. Tappin and Tao Ma and Peter S. Park and Rayan Onyonka and Arthur Hjorth and Peter Slattery and Qingcheng Zeng and Lennart Finke and Igor Grossmann and Alessandro Salatiello and Ezra Karger},
journal= {arXiv preprint arXiv:2505.09662},
year = {2026}
}