English
Related papers

Related papers: Evaluating the Effectiveness of Large Language Mod…

200 papers

With the rapid advance of machine learning (ML) technology, large language models (LLMs) are increasingly explored as an intelligent tool to generate program code from natural language specifications. However, existing evaluations of LLMs…

Software Engineering · Computer Science 2024-06-19 Tanha Miah , Hong Zhu

Large language models (LLMs) such as ChatGPT can produce coherent, cohesive, relevant, and fluent answers for various natural language processing (NLP) tasks. Taking document-level machine translation (MT) as a testbed, this paper provides…

Computation and Language · Computer Science 2023-10-25 Longyue Wang , Chenyang Lyu , Tianbo Ji , Zhirui Zhang , Dian Yu , Shuming Shi , Zhaopeng Tu

Large language models (LLMs) are increasingly used as conversational partners for learning, yet the interactional dynamics supporting users' learning and engagement are understudied. We analyze the linguistic and interactional features from…

Computation and Language · Computer Science 2026-03-13 Shaz Furniturewala , Gerard Christopher Yeo , Kokil Jaidka

Large language models (LLMs) and prompt engineering hold significant potential for advancing computer programming education through personalized instruction. This paper explores this potential by investigating three critical research…

Artificial Intelligence · Computer Science 2024-07-09 Tianyu Wang , Nianjun Zhou , Zhixiong Chen

Effectively supporting students in mastering all facets of self-regulated learning is a central aim of teachers and educational researchers. Prior research could demonstrate that formative feedback is an effective way to support students…

Physics Education · Physics 2024-12-31 Steffen Steinert , Karina E. Avila , Stefan Ruzika , Jochen Kuhn , Stefan Küchemann

This paper was a Workshop Paper. See the full paper which will be presented at CHI 2026: arXiv:2501.10551; As large language models (LLMs) become more powerful and ubiquitous, systems like ChatGPT are increasingly used by students to help…

Human-Computer Interaction · Computer Science 2026-02-13 Andrew Jelson , Sang Won Lee

Large Language Models (LLMs) are conversational interfaces. As such, LLMs have the potential to assist their users not only when they can fully specify the task at hand, but also to help them define, explore, and refine what they need…

Computation and Language · Computer Science 2025-05-12 Philippe Laban , Hiroaki Hayashi , Yingbo Zhou , Jennifer Neville

Traditional interventions for academic procrastination often fail to capture the nuanced, individual-specific factors that underlie them. Large language models (LLMs) hold immense potential for addressing this gap by permitting open-ended…

Providing scaffolding through educational chatbots built on Large Language Models (LLM) has potential risks and benefits that remain an open area of research. When students navigate impasses, they ask for help by formulating impasse-driven…

Human-Computer Interaction · Computer Science 2026-02-23 Alexandra Neagu , Marcus Messer , Peter Johnson , Rhodri Nelson

Recent advances in large language models (LLMs) demonstrate their potential as educational tutors. However, different tutoring strategies benefit different student personalities, and mismatches can be counterproductive to student outcomes.…

Computation and Language · Computer Science 2026-01-14 Donya Rooein , Sankalan Pal Chowdhury , Mariia Eremeeva , Yuan Qin , Debora Nozza , Mrinmaya Sachan , Dirk Hovy

The rapid adoption of Generative AI (GenAI) based on Large Language Models (LLMs) such as ChatGPT has recently and profoundly impacted education, offering transformative opportunities while raising significant concerns. In this study we…

Computers and Society · Computer Science 2024-12-24 Jérémie Sublime , Ilaria Renna

Large language models (LLMs) are now deployed at unprecedented scale, assisting millions of users in daily tasks. However, the risk of these models assisting unlawful activities remains underexplored. In this study, we define this high-risk…

Computers and Society · Computer Science 2025-11-27 Xing Wang , Huiyuan Xie , Yiyan Wang , Chaojun Xiao , Huimin Chen , Holli Sargeant , Felix Steffek , Jie Shao , Zhiyuan Liu , Maosong Sun

Automatic evaluation is an integral aspect of dialogue system research. The traditional reference-based NLG metrics are generally found to be unsuitable for dialogue assessment. Consequently, recent studies have suggested various unique,…

Computation and Language · Computer Science 2024-01-23 Chen Zhang , Luis Fernando D'Haro , Yiming Chen , Malu Zhang , Haizhou Li

Reciprocal questioning is essential for effective teaching and learning, fostering active engagement and deeper understanding through collaborative interactions, especially in large classrooms. Can large language model (LLM), such as…

Computers and Society · Computer Science 2023-11-28 Chee Wei Tan

This study investigates the potential for Large Language Models (LLMs) to scale-up Dynamic Assessment (DA). To facilitate such an investigation, we first developed DynaWrite-a modular, microservices-based grammatical tutoring application…

Computation and Language · Computer Science 2025-09-08 Timur Jaganov , John Blake , Julián Villegas , Nicholas Carr

This study evaluates the effectiveness of various large language models (LLMs) in performing tasks common among undergraduate computer science students. Although a number of research studies in the computing education community have…

Computers and Society · Computer Science 2024-04-04 Vibhor Agarwal , Madhav Krishan Garg , Sahiti Dharmavaram , Dhruv Kumar

This paper presents an integrated systematic study of the performance of large language models (LLMs), specifically ChatGPT, for automatically formulating and solving Stochastic Optimization (SO) problems from natural language descriptions.…

Artificial Intelligence · Computer Science 2026-01-15 Amirreza Talebi

Large language models (LLMs) are increasingly used in robotics, especially for high-level action planning. Meanwhile, many robotics applications involve human supervisors or collaborators. Hence, it is crucial for LLMs to generate socially…

Robotics · Computer Science 2025-05-28 Lennart Wachowiak , Andrew Coles , Oya Celiktutan , Gerard Canal

Recently, large language models (LLMs), such as GPT-4, stand out remarkable conversational abilities, enabling them to engage in dynamic and contextually relevant dialogues across a wide range of topics. However, given a long conversation,…

Computation and Language · Computer Science 2025-08-26 Qingyue Wang , Yanhe Fu , Yanan Cao , Shuai Wang , Zhiliang Tian , Liang Ding

Large Language Models (LLMs) have upended decades of pedagogy in computing education. Students previously learned to code through \textit{writing} many small problems with less emphasis on code reading and comprehension. Recent research has…