English
Related papers

Related papers: An Empirical Study on Challenges for LLM Applicati…

200 papers

The adoption of large language models (LLMs) and autonomous agents in software engineering marks an enduring paradigm shift. These systems create new opportunities for tool design, workflow orchestration, and empirical observation, while…

Software Engineering · Computer Science 2025-11-04 Christoph Treude , Margaret-Anne Storey

While large language models (LLMs) like ChatGPT have shown impressive capabilities in Natural Language Processing (NLP) tasks, a systematic investigation of their potential in this field remains largely unexplored. This study aims to…

Computation and Language · Computer Science 2025-08-26 Libo Qin , Qiguang Chen , Xiachong Feng , Yang Wu , Yongheng Zhang , Yinghui Li , Min Li , Wanxiang Che , Philip S. Yu

Since the launch of ChatGPT, a powerful AI Chatbot developed by OpenAI, large language models (LLMs) have made significant advancements in both academia and industry, bringing about a fundamental engineering paradigm shift in many areas.…

Software Engineering · Computer Science 2023-11-22 Xiaoxia Liu , Jingyi Wang , Jun Sun , Xiaohan Yuan , Guoliang Dong , Peng Di , Wenhai Wang , Dongxia Wang

Large Language Models (LLMs) have gained considerable traction within the Software Engineering (SE) community, impacting various SE tasks from code completion to test generation, from program repair to code summarization. Despite their…

Software Engineering · Computer Science 2024-01-09 June Sallou , Thomas Durieux , Annibale Panichella

The adoption of Large Language Models (LLMs) is reshaping software development as developers integrate these LLMs into their applications. In such applications, prompts serve as the primary means of interacting with LLMs. Despite the…

Software Engineering · Computer Science 2025-01-14 Mahan Tafreshipour , Aaron Imani , Eric Huang , Eduardo Almeida , Thomas Zimmermann , Iftekhar Ahmed

This paper investigates the complexities of integrating Large Language Models (LLMs) into software products, with a focus on the challenges encountered for determining their readiness for release. Our systematic review of grey literature…

Software Engineering · Computer Science 2024-03-29 Harsh Patel , Dominique Boucher , Emad Fallahzadeh , Ahmed E. Hassan , Bram Adams

Large language models (LLMs), such as ChatGPT, have rapidly penetrated into people's work and daily lives over the past few years, due to their extraordinary conversational skills and intelligence. ChatGPT has become the fastest-growing…

Computation and Language · Computer Science 2024-09-04 Wenxuan Wang

The release of ChatGPT in late 2022 caused a flurry of activity and concern in the academic and educational communities. Some see the tool's ability to generate human-like text that passes at least cursory inspections for factual accuracy…

Computers and Society · Computer Science 2025-04-18 Leo Leppänen , Lili Aunimo , Arto Hellas , Jukka K. Nurminen , Linda Mannila

Coverage of ChatGPT-style large language models (LLMs) in the media has focused on their eye-catching achievements, including solving advanced mathematical problems and reaching expert proficiency in medical examinations. But the gradual…

Computers and Society · Computer Science 2024-03-26 Djavan De Clercq , Elias Nehring , Harry Mayne , Adam Mahdi

Large Language Models (LLMs) are used for many different software engineering tasks. In software architecture, they have been applied to tasks such as classification of design decisions, detection of design patterns, and generation of…

Software Engineering · Computer Science 2025-05-23 Larissa Schmid , Tobias Hey , Martin Armbruster , Sophie Corallo , Dominik Fuchß , Jan Keim , Haoyu Liu , Anne Koziolek

The proliferation of Large Language Models like ChatGPT has significantly advanced language understanding and generation, impacting a broad spectrum of applications. However, these models predominantly excel in text-based tasks, overlooking…

Computation and Language · Computer Science 2023-11-23 Xiao Liu , Jianfeng Lin , Jiawei Zhang

Newcomers onboarding to Open Source Software (OSS) projects face many challenges. Large Language Models (LLMs), like ChatGPT, have emerged as potential resources for answering questions and providing guidance, with many developers now…

Software Engineering · Computer Science 2025-02-12 Italo Santos , Katia Romero Felizardo , Igor Steinmacher , Marco A. Gerosa

Large Language Models (LLMs) have made significant progress in reasoning, demonstrating their capability to generate human-like responses. This study analyzes the problem-solving capabilities of LLMs in the domain of thermodynamics. A…

Computational Engineering, Finance, and Science · Computer Science 2025-12-18 Rebecca Loubet , Pascal Zittlau , Luisa Vollmer , Marco Hoffmann , Sophie Fellenz , Fabian Jirasek , Heike Leitte , Hans Hasse

Large language models (LLMs) demonstrate powerful information handling capabilities and are widely integrated into chatbot applications. OpenAI provides a platform for developers to construct custom GPTs, extending ChatGPT's functions and…

Cryptography and Security · Computer Science 2025-06-05 Wei Wenying , Zhao Kaifa , Xue Lei , Fan Ming

In this survey, we address the key challenges in Large Language Models (LLM) research, focusing on the importance of interpretability. Driven by increasing interest from AI and business sectors, we highlight the need for transparency in…

Computation and Language · Computer Science 2024-07-23 Erik Cambria , Lorenzo Malandri , Fabio Mercorio , Navid Nobani , Andrea Seveso

The application of Large Language Models (LLMs) in software engineering, particularly in static analysis tasks, represents a paradigm shift in the field. In this paper, we investigate the role that current LLMs can play in improving…

Software Engineering · Computer Science 2024-02-28 Ashwin Prasad Shivarpatna Venkatesh , Samkutty Sabu , Amir M. Mir , Sofia Reis , Eric Bodden

Recent advancements in artificial intelligence (AI) are fundamentally reshaping computing, with large language models (LLMs) now effectively being able to generate and interpret source code and natural language instructions. These emergent…

Applications of Large Language Models~(LLMs) have evolved from simple text generators into complex software systems that integrate retrieval augmentation, tool invocation, and multi-turn interactions. Their inherent non-determinism,…

Software Engineering · Computer Science 2025-08-29 Wei Ma , Yixiao Yang , Qiang Hu , Shi Ying , Zhi Jin , Bo Du , Zhenchang Xing , Tianlin Li , Junjie Shi , Yang Liu , Linxiao Jiang

Recent advances in AI, machine learning, and NLP have led to the development of a new generation of Large Language Models (LLMs) that are trained on massive amounts of data and often have trillions of parameters. Commercial applications…

Computation and Language · Computer Science 2024-04-05 Nishat Raihan , Dhiman Goswami , Sadiya Sayara Chowdhury Puspo , Christian Newman , Tharindu Ranasinghe , Marcos Zampieri

AI-assisted code generation tools have revolutionized software development, offering unprecedented efficiency and scalability. However, multiple studies have consistently highlighted challenges such as security vulnerabilities, reliability…

Software Engineering · Computer Science 2025-06-16 Ahilan Ayyachamy Nadar Ponnusamy