English
Related papers

Related papers: Why LLMs Fail: A Failure Analysis and Partial Succ…

200 papers

Fault Localization (FL) aims to automatically localize buggy lines of code, a key first step in many manual and automatic debugging tasks. Previous FL techniques assume the provision of input tests, and often require extensive program…

Software Engineering · Computer Science 2023-10-04 Aidan Z. H. Yang , Ruben Martins , Claire Le Goues , Vincent J. Hellendoorn

Timely and effective vulnerability patching is essential for cybersecurity defense, for which various approaches have been proposed yet still struggle to generate valid and correct patches for real-world vulnerabilities. In this paper, we…

Cryptography and Security · Computer Science 2025-04-04 Yu Nong , Haoran Yang , Long Cheng , Hongxin Hu , Haipeng Cai

Automated Program Repair (APR) has emerged as a promising paradigm for reducing debugging time and improving the overall efficiency of software development. Recent advances in Large Language Models (LLMs) have demonstrated their potential…

Software Engineering · Computer Science 2025-09-23 Shunyu Liu , Guangdong Bai , Mark Utting , Guowei Yang

As software becomes increasingly complex and prone to vulnerabilities, automated vulnerability detection is critically important, yet challenging. Given the significant successes of large language models (LLMs) in various tasks, there is…

Artificial Intelligence · Computer Science 2023-12-25 Zeyu Gao , Hao Wang , Yuchen Zhou , Wenyu Zhu , Chao Zhang

Automated Program Repair (APR) agents leverage Large Language Models (LLMs) to autonomously diagnose and fix software bugs through reasoning, planning, and tool use. Despite impressive leaderboard gains on benchmarks such as SWE-bench,…

Software Engineering · Computer Science 2026-05-28 Ira Ceka , Hailie Mitchell , Saurabh Pujar , Luca Buratti , Shyam Ramji , Junfeng Yang , Gail Kaiser , Baishakhi Ray

Generative Large Language Models (LLMs) are increasingly used in non-generative software maintenance tasks, such as fault localization (FL). Success in FL depends on a models ability to reason about program semantics beyond surface-level…

The significant advancements in Large Language Models (LLMs) have resulted in their widespread adoption across various tasks within Software Engineering (SE), including vulnerability detection and repair. Numerous studies have investigated…

Software Engineering · Computer Science 2024-10-08 Xin Zhou , Sicong Cao , Xiaobing Sun , David Lo

A key challenge in security analysis is the manual evaluation of potential security weaknesses generated by static application security testing (SAST) tools. Numerous false positives (FPs) in these reports reduce the effectiveness of…

Cryptography and Security · Computer Science 2025-07-15 Jonas Wagner , Simon Müller , Christian Näther , Jan-Philipp Steghöfer , Andreas Both

Deep learning-based approaches, particularly those leveraging pre-trained language models (PLMs), have shown promise in automated software vulnerability detection. However, existing methods are predominantly limited to specific programming…

Software Engineering · Computer Science 2025-05-13 Junji Yu , Honglin Shu , Michael Fu , Dong Wang , Chakkrit Tantithamthavorn , Yasutaka Kamei , Junjie Chen

The increasing adoption of Large Language Models (LLMs) in software engineering has sparked interest in their use for software vulnerability detection. However, the rapid development of this field has resulted in a fragmented research…

Software Engineering · Computer Science 2025-12-22 Sabrina Kaniewski , Fabian Schmidt , Markus Enzweiler , Michael Menth , Tobias Heer

This study presents a quantitative evaluation of the code quality and security of five prominent Large Language Models (LLMs): Claude Sonnet 4, Claude 3.7 Sonnet, GPT-4o, Llama 3.2 90B, and OpenCoder 8B. While prior research has assessed…

Software Engineering · Computer Science 2025-08-21 Abbas Sabra , Olivier Schmitt , Joseph Tyler

Large Language Models increasingly power critical infrastructure from healthcare to finance, yet their vulnerability to adversarial manipulation threatens system integrity and user safety. Despite growing deployment, no comprehensive…

Cryptography and Security · Computer Science 2026-03-19 Taiwo Onitiju , Iman Vakilinia

Automating hardware (HW) security vulnerability detection and mitigation during the design phase is imperative for two reasons: (i) It must be before chip fabrication, as post-fabrication fixes can be costly or even impractical; (ii) The…

Cryptography and Security · Computer Science 2024-04-26 Mohammad Akyash , Hadi Mardani Kamali

With the growing popularity of Large Language Models (LLMs) in software engineers' daily practices, it is important to ensure that the code generated by these tools is not only functionally correct but also free of vulnerabilities. Although…

Software Engineering · Computer Science 2024-09-06 Mohammed Latif Siddiq , Joanna C. S. Santos , Sajith Devareddy , Anna Muller

Providing effective feedback is important for student learning in programming problem-solving. In this sense, Large Language Models (LLMs) have emerged as potential tools to automate feedback generation. However, their reliability and…

Software Engineering · Computer Science 2025-03-20 Priscylla Silva , Evandro Costa

Software vulnerabilities pose significant security challenges and potential risks to society, necessitating extensive efforts in automated vulnerability detection. There are two popular lines of work to address automated vulnerability…

Software Engineering · Computer Science 2024-07-24 Xin Zhou , Duc-Manh Tran , Thanh Le-Cong , Ting Zhang , Ivana Clairine Irsan , Joshua Sumarlin , Bach Le , David Lo

Developers and organizations are using Large Language Models (LLMs) to generate security-critical code more frequently than ever, including cryptographic solutions for their products. This study presents an empirical evaluation of…

Cryptography and Security · Computer Science 2026-05-01 Mohamed Elsayed , Kenneth Fulton , Jeong Yang

Ensuring the security of large language models (LLMs) is an ongoing challenge despite their widespread popularity. Developers work to enhance LLMs security, but vulnerabilities persist, even in advanced versions like GPT-4. Attackers…

Cryptography and Security · Computer Science 2023-12-19 Aysan Esmradi , Daniel Wankit Yip , Chun Fai Chan

The rapid adoption of Large Language Models(LLMs) for code generation has transformed software development, yet little attention has been given to how security vulnerabilities evolve through iterative LLM feedback. This paper analyzes…

Software Engineering · Computer Science 2025-09-29 Shivani Shukla , Himanshu Joshi , Romilla Syed

Automated Program Repair (APR) plays a critical role in enhancing the quality and reliability of software systems. While substantial progress has been made in Java-based APR, largely facilitated by benchmarks like Defects4J, there remains a…

Software Engineering · Computer Science 2025-12-03 Jian Wang , Xiaofei Xie , Qiang Hu , Shangqing Liu , Jiongchi Yu , Jiaolong Kong , Yi Li