English
Related papers

Related papers: Large Language Models (GPT) Struggle to Answer Mul…

200 papers

The GPT (Generative Pre-trained Transformer) language models are an artificial intelligence and natural language processing technology that enables automatic text generation. There is a growing interest in applying GPT language models to…

Computers and Society · Computer Science 2024-03-25 Manuel de Buenaga , Francisco Javier Bueno

AI-powered education technologies can support students and teachers in computer science education. However, with the recent developments in generative AI, and especially the increasingly emerging popularity of ChatGPT, the effectiveness of…

Artificial Intelligence · Computer Science 2023-12-05 Nora Dunder , Saga Lundborg , Olga Viberg , Jacqueline Wong

Generative Pre-trained Transformer 4 (GPT-4) is the fourth-generation language model in the GPT series, developed by OpenAI, which promises significant advancements in the field of natural language processing (NLP). In this research…

Computation and Language · Computer Science 2023-05-08 Jawid Ahmad Baktash , Mursal Dawodi

Over the past decade, predictive language modeling for code has proven to be a valuable tool for enabling new forms of automation for developers. More recently, we have seen the advent of general purpose "large language models", based on…

Software Engineering · Computer Science 2023-06-05 Sajed Jalil , Suzzana Rafi , Thomas D. LaToza , Kevin Moran , Wing Lam

The rapid advancement of large language models, such as the Generative Pre-trained Transformer (GPT) series, has had significant implications across various disciplines. In this study, we investigate the potential of the state-of-the-art…

Computation and Language · Computer Science 2023-09-06 Yunhao Yang , Anshul Tomar

The study explores the capabilities of OpenAI's ChatGPT in solving different types of physics problems. ChatGPT (with GPT-4) was queried to solve a total of 40 problems from a college-level engineering physics course. These problems ranged…

Artificial Intelligence · Computer Science 2023-10-31 Karen D. Wang , Eric Burkholder , Carl Wieman , Shima Salehi , Nick Haber

One-on-one tutoring is widely acknowledged as an effective instructional method, conditioned on qualified tutors. However, the high demand for qualified tutors remains a challenge, often necessitating the training of novice tutors (i.e.,…

Computation and Language · Computer Science 2024-05-03 Jionghao Lin , Zifei Han , Danielle R. Thomas , Ashish Gurung , Shivang Gupta , Vincent Aleven , Kenneth R. Koedinger

A less complex and more straightforward program is a crucial factor that enhances its maintainability and makes writing secure and bug-free programs easier. However, due to its heavy workload and the risks of breaking the working programs,…

Programming Languages · Computer Science 2024-04-08 Atsushi Shirafuji , Yusuke Oda , Jun Suzuki , Makoto Morishita , Yutaka Watanobe

Generative AI and large language models have the potential to drastically improve the landscape of computing education by automatically generating personalized feedback and content. Recent works have studied the capabilities of these models…

Machine Learning · Computer Science 2023-08-08 Adish Singla

In this paper, military use cases or applications and implementation thereof are considered for natural language processing and large language models, which have broken into fame with the invention of the generative pre-trained transformer…

Computation and Language · Computer Science 2025-11-14 Satu Johansson , Taneli Riihonen

The rise of Generative AI (GenAI) tools like ChatGPT has created new opportunities and challenges for computing education. Existing research has primarily focused on GenAI's ability to complete educational tasks and its impact on student…

Software Engineering · Computer Science 2025-11-18 Rufeng Chen , Shuaishuai Jiang , Jiyun Shen , AJung Moon , Lili Wei

This paper presents an empirical evaluation of the performance of the Generative Pre-trained Transformer (GPT) model in Harvard's CS171 data visualization course. While previous studies have focused on GPT's ability to generate code for…

Human-Computer Interaction · Computer Science 2024-05-14 Chen Zhu-Tian , Chenyang Zhang , Qianwen Wang , Jakob Troidl , Simon Warchol , Johanna Beyer , Nils Gehlenborg , Hanspeter Pfister

While large language models (LLMs), such as GPT-3, appear to be robust and general, their reasoning ability is not at a level to compete with the best models trained for specific natural language reasoning problems. In this study, we…

Computation and Language · Computer Science 2023-07-18 Zhun Yang , Adam Ishay , Joohyung Lee

This study investigates how LLMs, specifically GPT-3.5 and GPT-4, can develop tailored questions for Grade 9 math, aligning with active learning principles. By utilizing an iterative method, these models adjust questions based on difficulty…

Computation and Language · Computer Science 2024-06-21 Hamdireza Rouzegar , Masoud Makrehchi

Generative Pre-trained Transformer (GPT) models have shown remarkable capabilities for natural language generation, but their performance for machine translation has not been thoroughly investigated. In this paper, we present a…

Generating multiple-choice questions (MCQs) with difficulty estimation remains challenging in automated MCQ-generation systems used in adaptive, AI-assisted education. This study proposes a novel methodology for generating MCQs with…

Computation and Language · Computer Science 2026-04-14 Mehmet Can Şakiroğlu , H. Altay Güvenir , Kamer Kaya

Evaluating the quality of automatically generated question items has been a long standing challenge. In this paper, we leverage LLMs to simulate student profiles and generate responses to multiple-choice questions (MCQs). The generative…

Human-Computer Interaction · Computer Science 2024-05-30 Xinyi Lu , Xu Wang

Lately, Large Language Models have been widely used in code generation. GPT4 is considered the most potent Large Language Model from Openai. In this paper, we examine GPT3.5 and GPT4 as coding assistants. More specifically, we have…

Artificial Intelligence · Computer Science 2023-09-25 Lefteris Moussiades , George Zografos

Students often do not fully understand the code they have written. This sometimes does not become evident until later in their education, which can mean it is harder to fix their incorrect knowledge or misunderstandings. In addition, being…

Software Engineering · Computer Science 2025-05-23 Martin Goodfellow , Robbie Booth , Andrew Fagan , Alasdair Lambert

Learning to use feature-rich software is a persistent challenge, but generative AI tools promise to lower this barrier by replacing complex navigation with natural language prompts. We investigated how people approach prompt-based tools for…

Human-Computer Interaction · Computer Science 2026-04-01 Kanak Gautam , Poorvi Bhatia , Parmit K. Chilana