English
Related papers

Related papers: Can we trust the evaluation on ChatGPT?

200 papers

With the release of ChatGPT and other large language models (LLMs) the discussion about the intelligence, possibilities, and risks, of current and future models have seen large attention. This discussion included much debated scenarios…

Artificial Intelligence · Computer Science 2024-07-31 Nils Körber , Silvan Wehrli , Christopher Irrgang

Recent development of large language models (LLMs), such as ChatGPT has been widely applied to a wide range of software engineering tasks. Many papers have reported their analysis on the potential advantages and limitations of ChatGPT for…

Software Engineering · Computer Science 2023-09-18 Priyanka Mudgal , Rita Wouhaybi

Large language models (LLMs) have become mainstream technology with their versatile use cases and impressive performance. Despite the countless out-of-the-box applications, LLMs are still not reliable. A lot of work is being done to improve…

Computation and Language · Computer Science 2023-06-13 Aisha Khatun , Daniel G. Brown

Large Language Models (LLMs) like ChatGPT have demonstrated amazing capabilities in comprehending user intents and generate reasonable and useful responses. Beside their ability to chat, their capabilities in various natural language…

Computation and Language · Computer Science 2026-02-12 Liz Li , Wei Zhu

As a way of addressing increasingly sophisticated problems, software professionals face the constant challenge of seeking improvement. However, for these individuals to enhance their skills, their process of studying and training must…

Software Engineering · Computer Science 2023-08-02 Gustavo Pinto , Isadora Cardoso-Pereira , Danilo Monteiro Ribeiro , Danilo Lucena , Alberto de Souza , Kiev Gama

ChatGPT has gained a huge popularity since its introduction. Its positive aspects have been reported through many media platforms, and some analyses even showed that ChatGPT achieved a decent grade in professional exams, adding extra…

Computation and Language · Computer Science 2023-11-15 Myeongjun Erik Jang , Thomas Lukasiewicz

Large-scale language models (LLMs) have emerged as a groundbreaking innovation in the realm of question-answering and conversational agents. These models, leveraging different deep learning architectures such as Transformers, are trained on…

Software Engineering · Computer Science 2023-07-18 Fardin Ahsan Sakib , Saadat Hasan Khan , A. H. M. Rezaul Karim

Scientific workflow systems are increasingly popular for expressing and executing complex data analysis pipelines over large datasets, as they offer reproducibility, dependability, and scalability of analyses by automatic parallelization on…

Distributed, Parallel, and Cluster Computing · Computer Science 2024-07-09 Mario Sänger , Ninon De Mecquenem , Katarzyna Ewa Lewińska , Vasilis Bountris , Fabian Lehmann , Ulf Leser , Thomas Kosch

While large language models (LLMs) like ChatGPT have shown impressive capabilities in Natural Language Processing (NLP) tasks, a systematic investigation of their potential in this field remains largely unexplored. This study aims to…

Computation and Language · Computer Science 2025-08-26 Libo Qin , Qiguang Chen , Xiachong Feng , Yang Wu , Yongheng Zhang , Yinghui Li , Min Li , Wanxiang Che , Philip S. Yu

The disruptive technology provided by large-scale pre-trained language models (LLMs) such as ChatGPT or GPT-4 has received significant attention in several application domains, often with an emphasis on high-level opportunities and…

Human-Computer Interaction · Computer Science 2023-06-27 Philippe J. Giabbanelli

Evaluating the quality of generated text is a challenging task in NLP, due to the inherent complexity and diversity of text. Recently, large language models (LLMs) have garnered significant attention due to their impressive performance in…

Computation and Language · Computer Science 2023-09-19 Yi Chen , Rui Wang , Haiyun Jiang , Shuming Shi , Ruifeng Xu

Large Language Models (LLMs) such as ChatGPT demonstrated the potential to replicate human language abilities through technology, ranging from text generation to engaging in conversations. However, it remains controversial to what extent…

Computation and Language · Computer Science 2025-07-09 Martin Schuele

The transformative influence of Large Language Models (LLMs) is profoundly reshaping the Artificial Intelligence (AI) technology domain. Notably, ChatGPT distinguishes itself within these models, demonstrating remarkable performance in…

Software Engineering · Computer Science 2024-05-28 Sayed Erfan Arefin , Tasnia Ashrafi Heya , Hasan Al-Qudah , Ynes Ineza , Abdul Serwadda

Recent advancements in Large Language Models (LLMs) have garnered wide attention and led to successful products such as ChatGPT and GPT-4. Their proficiency in adhering to instructions and delivering harmless, helpful, and honest (3H)…

Machine Learning · Computer Science 2023-10-11 Hao Sun

ChatGPT embarks on a new era of artificial intelligence and will revolutionize the way we approach intelligent traffic safety systems. This paper begins with a brief introduction about the development of large language models (LLMs). Next,…

Computation and Language · Computer Science 2023-09-07 Ou Zheng , Mohamed Abdel-Aty , Dongdong Wang , Zijin Wang , Shengxuan Ding

Large language model (LLM) applications, such as ChatGPT, are a powerful tool for online information-seeking (IS) and problem-solving tasks. However, users still face challenges initializing and refining prompts, and their cognitive…

Human-Computer Interaction · Computer Science 2024-02-12 Ben Wang , Jiqun Liu , Jamshed Karimnazarov , Nicolas Thompson

Large Language Models (LLMs) such as ChatGPT and its competitors have caused a revolution in natural language processing, but their capabilities also introduce new security vulnerabilities. This survey provides a comprehensive overview of…

Cryptography and Security · Computer Science 2025-08-26 Miles Q. Li , Benjamin C. M. Fung

ChatGPT has become a global sensation. As ChatGPT and other Large Language Models (LLMs) emerge, concerns of misusing them in various ways increase, such as disseminating fake news, plagiarism, manipulating public opinion, cheating, and…

Machine Learning · Computer Science 2023-04-06 Alessandro Pegoraro , Kavita Kumari , Hossein Fereidooni , Ahmad-Reza Sadeghi

While recent advancements in the capabilities and widespread accessibility of generative language models, such as ChatGPT (OpenAI, 2022), have brought about various benefits by generating fluent human-like text, the task of distinguishing…

Computation and Language · Computer Science 2023-09-15 Mahdi Dhaini , Wessel Poelman , Ege Erdogan

This paper investigates the performance of the Large Language Models (LLMs) ChatGPT-3.5 and GPT-4 in solving introductory programming tasks. Based on the performance, implications for didactic scenarios and assessment formats utilizing LLMs…

Software Engineering · Computer Science 2023-08-21 Natalie Kiesler , Daniel Schiffner