English
Related papers

Related papers: Can we trust the evaluation on ChatGPT?

200 papers

The use of ChatGPT and similar Large Language Model (LLM) tools in scholarly communication and academic publishing has been widely discussed since they became easily accessible to a general audience in late 2022. This study uses keywords…

Digital Libraries · Computer Science 2024-03-26 Andrew Gray

Learning from human feedback is a prominent technique to align the output of large language models (LLMs) with human expectations. Reinforcement learning from human feedback (RLHF) leverages human preference signals that are in the form of…

Computation and Language · Computer Science 2023-11-27 Di Jin , Shikib Mehri , Devamanyu Hazarika , Aishwarya Padmakumar , Sungjin Lee , Yang Liu , Mahdi Namazifar

Large Language Models (LLMs) such as ChatGPT can infer personal attributes from seemingly innocuous text, raising privacy risks beyond memorized data leakage. While prior work has demonstrated these risks, little is known about how users…

Human-Computer Interaction · Computer Science 2025-09-16 Synthia Wang , Sai Teja Peddinti , Nina Taft , Nick Feamster

Large Language Models (LLMs) have emerged as influential instruments within the realm of natural language processing; nevertheless, their capacity to handle multi-party conversations (MPCs) -- a scenario marked by the presence of multiple…

Computation and Language · Computer Science 2023-10-26 Chao-Hong Tan , Jia-Chen Gu , Zhen-Hua Ling

Large Language Models (LLMs) are advanced Artificial Intelligence (AI) systems that have undergone extensive training using large datasets in order to understand and produce language that closely resembles that of humans. These models have…

Software Engineering · Computer Science 2023-08-10 Alessio Buscemi

Recent advancements in large language models (LLMs) have led to the development of highly potent models like OpenAI's ChatGPT. These models have exhibited exceptional performance in a variety of tasks, such as question answering, essay…

Computation and Language · Computer Science 2023-04-12 Ruixiang Tang , Xiaotian Han , Xiaoqian Jiang , Xia Hu

Language models have steadily increased in size over the past few years. They achieve a high level of performance on various natural language processing (NLP) tasks such as question answering and summarization. Large language models (LLMs)…

Computation and Language · Computer Science 2023-01-31 Jessica Huynh , Cathy Jiao , Prakhar Gupta , Shikib Mehri , Payal Bajaj , Vishrav Chaudhary , Maxine Eskenazi

Large language models (LLMs) such as ChatGPT can produce coherent, cohesive, relevant, and fluent answers for various natural language processing (NLP) tasks. Taking document-level machine translation (MT) as a testbed, this paper provides…

Computation and Language · Computer Science 2023-10-25 Longyue Wang , Chenyang Lyu , Tianbo Ji , Zhirui Zhang , Dian Yu , Shuming Shi , Zhaopeng Tu

The introduction of ChatGPT has led to a significant increase in the utilization of Large Language Models (LLMs) for addressing downstream tasks. There's an increasing focus on cost-efficient training and deployment within this context.…

This paper examines the application of ChatGPT, a large language model (LLM), for question-and-answer (Q&A) tasks in the highly specialized field of nuclear data. The primary focus is on evaluating ChatGPT's performance on a curated test…

Computation and Language · Computer Science 2024-09-04 Muhammad Anwar , Mischa de Costa , Issam Hammad , Daniel Lau

Lab results are often confusing and hard to understand. Large language models (LLMs) such as ChatGPT have opened a promising avenue for patients to get their questions answered. We aim to assess the feasibility of using LLMs to generate…

Computation and Language · Computer Science 2024-04-23 Zhe He , Balu Bhasuran , Qiao Jin , Shubo Tian , Karim Hanna , Cindy Shavor , Lisbeth Garcia Arguello , Patrick Murray , Zhiyong Lu

Large language models (LLMs) have made significant progress in NLP. However, their ability to memorize, represent, and leverage commonsense knowledge has been a well-known pain point. In this paper, we specifically focus on ChatGPT, a…

Computation and Language · Computer Science 2024-04-22 Ning Bian , Xianpei Han , Le Sun , Hongyu Lin , Yaojie Lu , Ben He , Shanshan Jiang , Bin Dong

Large language models, pivotal in artificial intelligence, find diverse applications. ChatGPT (Chat Generative Pre-trained Transformer), an OpenAI creation, stands out as a widely adopted, powerful tool. It excels in chatbots, content…

Computation and Language · Computer Science 2025-05-28 Walid Hariri

Large language models (LLMs) have demonstrated remarkable capabilities across a wide range of tasks in various domains. Despite their impressive performance, they can be unreliable due to factual errors in their generations. Assessing their…

Computation and Language · Computer Science 2024-03-26 Jiahui Geng , Fengyu Cai , Yuxia Wang , Heinz Koeppl , Preslav Nakov , Iryna Gurevych

Simulation powered by Large Language Models (LLMs) has become a promising method for exploring complex human social behaviors. However, the application of LLMs in simulations presents significant challenges, particularly regarding their…

Computers and Society · Computer Science 2025-02-26 Qian Wang , Zhenheng Tang , Bingsheng He

The opacity in developing large language models (LLMs) is raising growing concerns about the potential contamination of public benchmarks in the pre-training data. Existing contamination detection methods are typically based on the text…

Computation and Language · Computer Science 2024-10-31 Feng Yao , Yufan Zhuang , Zihao Sun , Sunan Xu , Animesh Kumar , Jingbo Shang

Recent breakthroughs in natural language processing (NLP) have permitted the synthesis and comprehension of coherent text in an open-ended way, therefore translating the theoretical algorithms into practical applications. The large language…

Computation and Language · Computer Science 2023-05-30 Terry Yue Zhuo , Yujin Huang , Chunyang Chen , Zhenchang Xing

The startling success of ChatGPT and other large language models (LLMs) using transformer-based generative neural network architecture in applications such as natural language processing and image synthesis has many researchers excited…

Artificial Intelligence · Computer Science 2024-11-05 Venkat Venkatasubramanian , Arijit Chakraborty

This study compares the design practices and performance of ChatGPT 4.0, a large language model (LLM), against graduate engineering students in a 48-hour prototyping hackathon, based on a dataset comprising more than 100 prototypes. The LLM…

Human-Computer Interaction · Computer Science 2025-03-04 Daniel Nygård Ege , Henrik H. Øvrebø , Vegar Stubberud , Martin Francis Berg , Christer Elverum , Martin Steinert , Håvard Vestad

Large language models have been demonstrated to be valuable in different fields. ChatGPT, developed by OpenAI, has been trained using massive amounts of data and simulates human conversation by comprehending context and generating…

Computation and Language · Computer Science 2023-04-05 Ali Borji