English
Related papers

Related papers: Evaluating Large Language Models with Psychometric…

200 papers

State of the art large language models (LLMs) have shown impressive performance on a variety of benchmark tasks and are increasingly used as components in larger applications, where LLM-based predictions serve as proxies for human…

Computation and Language · Computer Science 2024-06-14 Michael Franke , Polina Tsvilodub , Fausto Carcassi

Large language models (LLMs) are gaining increasing popularity in both academia and industry, owing to their unprecedented performance in various applications. As LLMs continue to play a vital role in both research and daily use, their…

Computation and Language · Computer Science 2024-01-01 Yupeng Chang , Xu Wang , Jindong Wang , Yuan Wu , Linyi Yang , Kaijie Zhu , Hao Chen , Xiaoyuan Yi , Cunxiang Wang , Yidong Wang , Wei Ye , Yue Zhang , Yi Chang , Philip S. Yu , Qiang Yang , Xing Xie

This paper explores the intersection of psychology and artificial intelligence through the development and evaluation of specialized Large Language Models (LLMs). We introduce PsychoLex, a suite of resources designed to enhance LLMs'…

Computation and Language · Computer Science 2024-08-19 Mohammad Amin Abbasi , Farnaz Sadat Mirnezami , Hassan Naderi

Logical reasoning consistently plays a fundamental and significant role in the domains of knowledge engineering and artificial intelligence. Recently, Large Language Models (LLMs) have emerged as a noteworthy innovation in natural language…

Computation and Language · Computer Science 2024-09-17 Fangzhi Xu , Qika Lin , Jiawei Han , Tianzhe Zhao , Jun Liu , Erik Cambria

Knowing how test takers answer items in educational assessments is essential for test development, to evaluate item quality, and to improve test validity. However, this process usually requires extensive pilot studies with human…

Computation and Language · Computer Science 2025-06-12 Andreas Säuberli , Diego Frassinelli , Barbara Plank

An essential problem in artificial intelligence is whether LLMs can simulate human cognition or merely imitate surface-level behaviors, while existing datasets suffer from either synthetic reasoning traces or population-level aggregation,…

Computation and Language · Computer Science 2026-03-31 Yuxuan Gu , Lunjun Liu , Xiaocheng Feng , Kun Zhu , Weihong Zhong , Lei Huang , Bing Qin

Large language models (LLMs) are recognized as systems that closely mimic aspects of human intelligence. This capability has attracted attention from the social science community, who see the potential in leveraging LLMs to replace human…

Computers and Society · Computer Science 2025-03-04 Qiuejie Xie , Qiming Feng , Tianqi Zhang , Qingqiu Li , Linyi Yang , Yuejie Zhang , Rui Feng , Liang He , Shang Gao , Yue Zhang

Adult neurodivergence, including Attention-Deficit/Hyperactivity Disorder (ADHD), high-functioning Autism Spectrum Disorder (ASD), and Cognitive Disengagement Syndrome (CDS), is marked by substantial symptom overlap that limits the…

Neurons and Cognition · Quantitative Biology 2026-01-23 Francesco Chiappone , Davide Marocco , Nicola Milano

The pursuit of human-level artificial intelligence (AI) has significantly advanced the development of autonomous agents and Large Language Models (LLMs). LLMs are now widely utilized as decision-making agents for their ability to interpret…

The escalating global mental health crisis, marked by persistent treatment gaps, availability, and a shortage of qualified therapists, positions Large Language Models (LLMs) as a promising avenue for scalable support. While LLMs offer…

As large language models (LLMs) appear to behave increasingly human-like in text-based interactions, more and more researchers become interested in investigating personality in LLMs. However, the diversity of psychological personality…

Computation and Language · Computer Science 2024-06-26 Zhiyuan Wen , Yu Yang , Jiannong Cao , Haoming Sun , Ruosong Yang , Shuaiqi Liu

The rapid rise of Large Language Models (LLMs) has created new disruptive possibilities for persuasive communication, enabling fully-automated, personalized, and interactive content generation at an unprecedented scale. In this paper, we…

Computation and Language · Computer Science 2026-04-22 Sander Noels , Alexander Rogiers , Maarten Buyl , Tijl De Bie

This paper explores the advancements in making large language models (LLMs) more human-like. We focus on techniques that enhance natural language understanding, conversational coherence, and emotional intelligence in AI systems. The study…

Computation and Language · Computer Science 2026-02-03 Ethem Yağız Çalık , Talha Rüzgar Akkuş

One open question in the study of Large Language Models (LLMs) is whether they can emulate human ethical reasoning and act as believable proxies for human judgment. To investigate this, we introduce a benchmark dataset comprising 196…

Computation and Language · Computer Science 2025-05-14 Jiashen , Du , Jesse Yao , Allen Liu , Zhekai Zhang

Large language models (LLMs) are increasingly used in human-AI interaction research and practice, yet existing capability and safety benchmarks reveal little about the value priorities these systems express or how those priorities…

Artificial Intelligence · Computer Science 2026-05-19 Gabriel Rongyang Lau , Wei Yan Low , Seow Min Koh , Fiona Fui-Hoon Nah , Andree Hartanto

Large language models (LLMs) are used as "digital twins" to replace human respondents, yet their psychometric comparability to humans is uncertain. We propose a construct-validity framework spanning construct representation and the…

Computers and Society · Computer Science 2026-01-23 Yufei Zhang , Zhihao Ma

Users of Large Language Models (LLMs) often perceive these models as intelligent entities with human-like capabilities. However, the extent to which LLMs' capabilities truly approximate human abilities remains a topic of debate. In this…

Computation and Language · Computer Science 2025-04-18 Mingrui Zan , Yunquan Zhang , Boyang Zhang , Fangming Liu , Daning Cheng

Research on emergent patterns in Large Language Models (LLMs) has gained significant traction in both psychology and artificial intelligence, motivating the need for a comprehensive review that offers a synthesis of this complex landscape.…

Computation and Language · Computer Science 2024-12-23 Zhisheng Tang , Mayank Kejriwal

Recent advances in Large Language Models (LLMs) highlight the need to align their behaviors with human values. A critical, yet understudied, issue is the potential divergence between an LLM's stated preferences (its reported alignment with…

Artificial Intelligence · Computer Science 2025-06-03 Zhuojun Gu , Quan Wang , Shuchu Han

Large Language Models (LLMs) are increasingly employed for simulating human behaviors across diverse domains. However, our position is that current LLM-based human simulations remain insufficiently reliable, as evidenced by significant…

Computation and Language · Computer Science 2025-12-02 Qian Wang , Jiaying Wu , Zichen Jiang , Zhenheng Tang , Bingqiao Luo , Nuo Chen , Wei Chen , Bingsheng He