English
Related papers

Related papers: Rethinking Role-Playing Evaluation: Anonymous Benc…

200 papers

Role-playing agents (RPA) have been a popular application area for large language models (LLMs), attracting significant interest from both industry and academia.While existing RPAs well portray the characters' knowledge and tones, they face…

Computation and Language · Computer Science 2024-10-16 Yiting Ran , Xintao Wang , Rui Xu , Xinfeng Yuan , Jiaqing Liang , Deqing Yang , Yanghua Xiao

Recent advancements in large language models (LLMs) have significantly boosted the rise of Role-Playing Language Agents (RPLAs), i.e., specialized AI systems designed to simulate assigned personas. By harnessing multiple advanced abilities…

Role-playing agents (RPAs), powered by large language models, have emerged as a flourishing field of applications. However, a key challenge lies in assessing whether RPAs accurately reproduce the personas of target characters, namely their…

Computation and Language · Computer Science 2024-06-10 Xintao Wang , Yunze Xiao , Jen-tse Huang , Siyu Yuan , Rui Xu , Haoran Guo , Quan Tu , Yaying Fei , Ziang Leng , Wei Wang , Jiangjie Chen , Cheng Li , Yanghua Xiao

The low-intrusion and automated personality assessment is receiving increasing attention in psychology and human-computer interaction fields. This study explores an interactive approach for personality assessment, focusing on the…

Human-Computer Interaction · Computer Science 2026-04-07 Baiqiao Zhang , Xiangxian Li , Chao Zhou , Xinyu Gai , Juan Liu , Xue Yang , Nianlong Li , Shuai Ma , Xiaojuan Ma , Yong-jin Liu , Yulong Bian

Large language models (LLMs) have demonstrated impressive performance and spurred numerous AI applications, in which role-playing agents (RPAs) are particularly popular, especially for fictional characters. The prerequisite for these RPAs…

Computation and Language · Computer Science 2024-10-10 Xinfeng Yuan , Siyu Yuan , Yuhan Cui , Tianhe Lin , Xintao Wang , Rui Xu , Jiangjie Chen , Deqing Yang

Large Language Models (LLMs) are increasingly deployed as autonomous agents capable of actions with real-world impacts beyond text generation. While persona-induced biases in text generation are well documented, their effects on agent task…

Computation and Language · Computer Science 2026-02-16 Linbo Cao , Lihao Sun , Yang Yue

Large language models (LLMs) increasingly serve as interactive social agents, yet their ability to maintain coherent and authentic persona-level role-playing remains limited, particularly in realistic social scenarios. Existing research…

Artificial Intelligence · Computer Science 2026-05-19 Wenlong Shi , Jianxun Lian , Mingqi Wu , Haiming Qin , Mingyang Zhou , Xing Xie , Naipeng Chao , Hao Liao

Role-Playing Agent (RPA) is an increasingly popular type of LLM Agent that simulates human-like behaviors in a variety of tasks. However, evaluating RPAs is challenging due to diverse task requirements and agent designs. This paper proposes…

Human-Computer Interaction · Computer Science 2025-03-28 Chaoran Chen , Bingsheng Yao , Ruishi Zou , Wenyue Hua , Weimin Lyu , Yanfang Ye , Toby Jia-Jun Li , Dakuo Wang

With the rapid improvement in the general capabilities of LLMs, LLM personalization, i.e., how to build LLM systems that can generate personalized responses or services that are tailored to distinct user personas, has become an increasingly…

Computation and Language · Computer Science 2025-06-17 Meiling Tao , Chenghao Zhu , Dongyi Ding , Tiannan Wang , Yuchen Eleanor Jiang , Wangchunshu Zhou

Recent privacy research on large language models (LLMs) has shown that they achieve near-human-level performance at inferring personal data from online texts. With ever-increasing model capabilities, existing text anonymization methods are…

Artificial Intelligence · Computer Science 2025-02-04 Robin Staab , Mark Vero , Mislav Balunović , Martin Vechev

Large language models (LLMs) have shown remarkable promise in simulating human language and behavior. This study investigates how integrating persona variables-demographic, social, and behavioral factors-impacts LLMs' ability to simulate…

Computation and Language · Computer Science 2024-06-18 Tiancheng Hu , Nigel Collier

This research focuses on using large language models (LLMs) to simulate social experiments, exploring their ability to emulate human personality in virtual persona role-playing. The research develops an end-to-end evaluation framework,…

Computers and Society · Computer Science 2025-10-15 Yuqi Bai , Tianyu Huang , Kun Sun , Yuting Chen

The advancement of Large Language Models (LLMs) has spurred significant interest in Role-Playing Agents (RPAs) for applications such as emotional companionship and virtual interaction. However, recent RPAs are often built on explicit…

Computation and Language · Computer Science 2025-06-03 Yihong Tang , Kehai Chen , Muyun Yang , Zhengyu Niu , Jing Li , Tiejun Zhao , Min Zhang

Current benchmarks for evaluating Large Language Models (LLMs) often do not exhibit enough writing style diversity, with many adhering primarily to standardized conventions. Such benchmarks do not fully capture the rich variety of…

Computation and Language · Computer Science 2025-09-29 Kimberly Le Truong , Riccardo Fogliato , Hoda Heidari , Zhiwei Steven Wu

Personalized opinion summarization is crucial as it considers individual user interests while generating product summaries. Recent studies show that although large language models demonstrate powerful text summarization and evaluation…

Computation and Language · Computer Science 2025-03-04 Yanyue Zhang , Yulan He , Deyu Zhou

In recent years, large language models (LLMs) have achieved breakthrough progress in many dialogue generation tasks. However, their lack of emotion and fine-grained role awareness limits the model's ability to provide personalized and…

Computation and Language · Computer Science 2025-03-26 Ke Ji , Yixin Lian , Linxu Li , Jingsheng Gao , Weiyuan Li , Bin Dai

We present a novel approach for enhancing diversity and control in data annotation tasks by personalizing large language models (LLMs). We investigate the impact of injecting diverse persona descriptions into LLM prompts across two studies,…

Computation and Language · Computer Science 2024-10-16 Leon Fröhling , Gianluca Demartini , Dennis Assenmacher

In recent years, with the rapid advancement of large language models (LLMs), role-playing language agents (RPLAs) have emerged as a prominent research focus at the intersection of natural language processing (NLP) and human-computer…

Computation and Language · Computer Science 2026-01-16 Ye Wang , Jiaxing Chen , Hongjiang Xiao

Large language models (LLMs) are increasingly being used in human-centered social scientific tasks, such as data annotation, synthetic data creation, and engaging in dialog. However, these tasks are highly subjective and dependent on human…

Computation and Language · Computer Science 2024-10-18 Salvatore Giorgi , Tingting Liu , Ankit Aich , Kelsey Isman , Garrick Sherman , Zachary Fried , João Sedoc , Lyle H. Ungar , Brenda Curtis

As language models achieve increasingly human-like capabilities in conversational text generation, a critical question emerges: to what extent can these systems simulate the characteristics of specific individuals? To evaluate this, we…

Computation and Language · Computer Science 2025-06-04 Quan Shi , Carlos E. Jimenez , Stephen Dong , Brian Seo , Caden Yao , Adam Kelch , Karthik Narasimhan
‹ Prev 1 2 3 10 Next ›