English
Related papers

Related papers: Evaluating Large Language Model Capabilities in As…

200 papers

Large Language Models (LLMs) have demonstrated exceptional capabilities in solving various tasks, progressively evolving into general-purpose assistants. The increasing integration of LLMs into society has sparked interest in whether they…

Computation and Language · Computer Science 2025-10-20 Yuan Li , Yue Huang , Hongyi Wang , Ying Cheng , Xiangliang Zhang , James Zou , Lichao Sun

Large Language Models (LLMs) have garnered considerable interest within both academic and industrial. Yet, the application of LLMs to graph data remains under-explored. In this study, we evaluate the capabilities of four LLMs in addressing…

Artificial Intelligence · Computer Science 2023-09-12 Chang Liu , Bo Wu

The use of large language models (LLMs) for qualitative analysis is gaining attention in various fields, including software engineering, where qualitative methods are essential for understanding human and social factors. This study aimed to…

Software Engineering · Computer Science 2025-03-11 Matheus de Morais Leça , Lucas Valença , Reydne Santos , Ronnie de Souza Santos

Large Language Models (LLMs) have made significant progress in reasoning, demonstrating their capability to generate human-like responses. This study analyzes the problem-solving capabilities of LLMs in the domain of thermodynamics. A…

Computational Engineering, Finance, and Science · Computer Science 2025-12-18 Rebecca Loubet , Pascal Zittlau , Luisa Vollmer , Marco Hoffmann , Sophie Fellenz , Fabian Jirasek , Heike Leitte , Hans Hasse

This study explores the potential of Large Language Models (LLMs), specifically GPT-4, to enhance objectivity in organizational task performance evaluations. Through comparative analyses across two studies, including various task…

Computation and Language · Computer Science 2024-08-13 Ning Li , Huaikang Zhou , Mingze Xu

In this paper, we explore the challenges inherent to Large Language Models (LLMs) like GPT-4, particularly their propensity for hallucinations, logic mistakes, and incorrect conclusions when tasked with answering complex questions. The…

Computation and Language · Computer Science 2023-12-22 Xiang Li , Haoran Tang , Siyu Chen , Ziwei Wang , Anurag Maravi , Marcin Abram

Logical reasoning consistently plays a fundamental and significant role in the domains of knowledge engineering and artificial intelligence. Recently, Large Language Models (LLMs) have emerged as a noteworthy innovation in natural language…

Computation and Language · Computer Science 2024-09-17 Fangzhi Xu , Qika Lin , Jiawei Han , Tianzhe Zhao , Jun Liu , Erik Cambria

Large Language Models (LLMs) have emerged as powerful tools in various research domains. This article examines their potential through a literature review and firsthand experimentation. While LLMs offer benefits like cost-effectiveness and…

Human-Computer Interaction · Computer Science 2024-04-10 M. Namvarpour , A. Razi

This study investigates the reliability and validity of five advanced Large Language Models (LLMs), Claude 3.5, DeepSeek v2, Gemini 2.5, GPT-4, and Mistral 24B, for automated essay scoring in a real world higher education context. A total…

Computers and Society · Computer Science 2025-08-05 Andrea Gaggioli , Giuseppe Casaburi , Leonardo Ercolani , Francesco Collova' , Pietro Torre , Fabrizio Davide

This paper investigates the capabilities of large language models (LLMs) in formulating and solving decision-making problems using mathematical programming. We first conduct a systematic review and meta-analysis of recent literature to…

Artificial Intelligence · Computer Science 2025-08-26 Mohammad J. Abdel-Rahman , Yasmeen Alslman , Dania Refai , Amro Saleh , Malik A. Abu Loha , Mohammad Yahya Hamed

Given the rapid ascent of large language models (LLMs), we study the question: (How) can large language models help in reviewing of scientific papers or proposals? We first conduct some pilot studies where we find that (i) GPT-4 outperforms…

Computation and Language · Computer Science 2023-06-02 Ryan Liu , Nihar B. Shah

Large Language Models (LLMs) are increasingly explored for educational tasks such as grading, yet their alignment with human evaluation in real classrooms remains underexamined. In this study, we investigate the feasibility of using an LLM…

Computation and Language · Computer Science 2025-11-19 Grace Byun , Swati Rajwal , Jinho D. Choi

Large language models (LLMs), especially when instruction-tuned for chat, have become part of our daily lives, freeing people from the process of searching, extracting, and integrating information from multiple sources by offering a…

Computation and Language · Computer Science 2024-11-01 Yuxia Wang , Minghan Wang , Muhammad Arslan Manzoor , Fei Liu , Georgi Georgiev , Rocktim Jyoti Das , Preslav Nakov

Large Language Models (LLMs) excel in various Natural Language Processing (NLP) tasks, yet their evaluation, particularly in languages beyond the top $20$, remains inadequate due to existing benchmarks and metrics limitations. Employing…

Computation and Language · Computer Science 2024-02-14 Rishav Hada , Varun Gumma , Adrian de Wynter , Harshita Diddee , Mohamed Ahmed , Monojit Choudhury , Kalika Bali , Sunayana Sitaram

Large language models (LLMs) have demonstrated their potential in social science research by emulating human perceptions and behaviors, a concept referred to as algorithmic fidelity. This study assesses the algorithmic fidelity and bias of…

Artificial Intelligence · Computer Science 2024-08-09 S. Lee , T. Q. Peng , M. H. Goldberg , S. A. Rosenthal , J. E. Kotcher , E. W. Maibach , A. Leiserowitz

The rapid advancement of artificial intelligence, particularly with the development of Large Language Models (LLMs) built on the transformer architecture, has redefined the capabilities of natural language processing. These models now…

Computation and Language · Computer Science 2025-02-11 Andrea Matarazzo , Riccardo Torlone

Large language models (LLMs) have demonstrated remarkable capabilities across a broad spectrum of tasks. They have attracted significant attention and been deployed in numerous downstream applications. Nevertheless, akin to a double-edged…

Computation and Language · Computer Science 2023-11-28 Zishan Guo , Renren Jin , Chuang Liu , Yufei Huang , Dan Shi , Supryadi , Linhao Yu , Yan Liu , Jiaxuan Li , Bojian Xiong , Deyi Xiong

Large Language Models (LLMs) have demonstrated exceptional capabilities in various natural language tasks, often achieving performances that surpass those of humans. Despite these advancements, the domain of mathematics presents a…

Computation and Language · Computer Science 2024-04-02 Ankit Satpute , Noah Giessing , Andre Greiner-Petter , Moritz Schubotz , Olaf Teschke , Akiko Aizawa , Bela Gipp

This paper investigates the utilization of Large Language Models (LLMs) for solving complex linguistic puzzles, a domain requiring advanced reasoning and adept translation capabilities akin to human cognitive processes. We explore specific…

Computation and Language · Computer Science 2025-02-04 Zheng-Lin Lin , Yu-Fei Shih , Shu-Kai Hsieh

The real estate market is vital to global economies but suffers from significant information asymmetry. This study examines how Large Language Models (LLMs) can democratize access to real estate insights by generating competitive and…

Artificial Intelligence · Computer Science 2025-10-01 Margot Geerts , Manon Reusens , Bart Baesens , Seppe vanden Broucke , Jochen De Weerdt