English
Related papers

Related papers: Evaluating Text Summaries Generated by Large Langu…

200 papers

As Large Language Models (LLMs) have become integral to both research and daily operations, rigorous evaluation is crucial. This assessment is important not only for individual tasks but also for understanding their societal impact and…

Software Engineering · Computer Science 2024-04-02 Zeeshan Rasheed , Muhammad Waseem , Kari Systä , Pekka Abrahamsson

ChatGPT has the ability to generate grammatically flawless and seemingly-human replies to different types of questions from various domains. The number of its users and of its applications is growing at an unprecedented rate. Unfortunately,…

Computation and Language · Computer Science 2023-02-01 Sandra Mitrović , Davide Andreoletti , Omran Ayoub

Drawing from the resources of psychoanalysis and critical media studies, in this paper we develop an analysis of Large Language Models (LLMs) as automated subjects. We argue the intentional fictional projection of subjectivity onto LLMs can…

Computers and Society · Computer Science 2022-12-13 Liam Magee , Vanicka Arora , Luke Munn

Human-like large language models (LLMs), especially the most powerful and popular ones in OpenAI's GPT family, have proven to be very helpful for many natural language processing (NLP) related tasks. Therefore, various attempts have been…

Computation and Language · Computer Science 2024-09-11 Ridong Han , Chaohao Yang , Tao Peng , Prayag Tiwari , Xiang Wan , Lu Liu , Benyou Wang

Human-like large language models (LLMs), especially the most powerful and popular ones in OpenAI's GPT family, have proven to be very helpful for many natural language processing (NLP) related tasks. Therefore, various attempts have been…

Computation and Language · Computer Science 2024-09-12 Ridong Han , Chaohao Yang , Tao Peng , Prayag Tiwari , Xiang Wan , Lu Liu , Benyou Wang

The success of Large Language Models (LLMs) in other domains has raised the question of whether LLMs can reliably assess and manipulate the readability of text. We approach this question empirically. First, using a published corpus of 4,724…

Computation and Language · Computer Science 2024-10-21 Sean Trott , Pamela D. Rivière

The rapid advancement of large language models (LLMs) has led to increasingly human-like AI-generated text, raising concerns about content authenticity, misinformation, and trustworthiness. Addressing the challenge of reliably detecting…

This study aims to explore the performance improvement method of large language models based on GPT-4 under the multi-task learning framework and conducts experiments on two tasks: text classification and automatic summary generation.…

Computation and Language · Computer Science 2024-12-10 Zhen Qi , Jiajing Chen , Shuo Wang , Bingying Liu , Hongye Zheng , Chihang Wang

Reliable evaluation of large language model (LLM)-generated summaries remains an open challenge, particularly across heterogeneous domains and document lengths. We conduct a comprehensive meta-evaluation of 14 automatic summarization…

Computation and Language · Computer Science 2026-04-29 Huyen Nguyen , Haoxuan Zhang , Yang Zhang , Junhua Ding , Haihua Chen

This study introduces and evaluates the PRECISE framework, utilizing OpenAI's GPT-4 to enhance patient engagement by providing clearer and more accessible chest X-ray reports at a sixth-grade reading level. The framework was tested on 500…

Recent advancements in Large Language Models (LLMs), particularly those built on Transformer architectures, have significantly broadened the scope of natural language processing (NLP) applications, transcending their initial use in chatbot…

Computation and Language · Computer Science 2024-05-29 Chen Wang , Jin Zhao , Jiaqi Gong

By harnessing pre-trained language models, summarization models had rapid progress recently. However, the models are mainly assessed by automatic evaluation metrics such as ROUGE. Although ROUGE is known for having a positive correlation…

Computation and Language · Computer Science 2021-06-03 Wonjin Yoon , Yoon Sun Yeo , Minbyul Jeong , Bong-Jun Yi , Jaewoo Kang

Recent advancements in large language models (LLMs) have led to the development of highly potent models like OpenAI's ChatGPT. These models have exhibited exceptional performance in a variety of tasks, such as question answering, essay…

Computation and Language · Computer Science 2023-04-12 Ruixiang Tang , Xiaotian Han , Xiaoqian Jiang , Xia Hu

A brief, fluent, and relevant summary can be helpful during program comprehension; however, such a summary does require significant human effort to produce. Often, good summaries are unavailable in software projects, which makes maintenance…

Software Engineering · Computer Science 2025-06-03 Yuvraj Virk , Premkumar Devanbu , Toufique Ahmed

Lately, Large Language Models have been widely used in code generation. GPT4 is considered the most potent Large Language Model from Openai. In this paper, we examine GPT3.5 and GPT4 as coding assistants. More specifically, we have…

Artificial Intelligence · Computer Science 2023-09-25 Lefteris Moussiades , George Zografos

In medical dialogue summarization, summaries must be coherent and must capture all the medically relevant information in the dialogue. However, learning effective models for summarization require large amounts of labeled data which is…

Computation and Language · Computer Science 2021-10-15 Bharath Chintagunta , Namit Katariya , Xavier Amatriain , Anitha Kannan

Automatic Text Summarization (ATS) is becoming relevant with the growth of textual data; however, with the popularization of public large-scale datasets, some recent machine learning approaches have focused on dense models and architectures…

Computation and Language · Computer Science 2023-03-07 Vinícius Camargo da Silva , João Paulo Papa , Kelton Augusto Pontara da Costa

Classifying policy documents into policy issue topics has been a long-time effort in political science and communication disciplines. Efforts to automate text classification processes for social science research purposes have so far…

Computation and Language · Computer Science 2023-10-13 Erkan Gunes , Christoffer Koch Florczak

The conversational artificial-intelligence (AI) technology ChatGPT has become one of the most widely used natural language processing tools. With thousands of published papers demonstrating its applications across various industries and…

Computers and Society · Computer Science 2023-05-30 Irene S. Gabashvili