English
Related papers

Related papers: Beyond Single Bugs: Benchmarking Large Language Mo…

200 papers

Large Language Models (LLM) continue to demonstrate their utility in a variety of emergent capabilities in different fields. An area that could benefit from effective language understanding in cybersecurity is the analysis of log files.…

Networking and Internet Architecture · Computer Science 2023-11-27 Egil Karlsen , Xiao Luo , Nur Zincir-Heywood , Malcolm Heywood

Large language models have shown good potential in supporting software development tasks. This is why more and more developers turn to LLMs (e.g. ChatGPT) to support them in fixing their buggy code. While this can save time and effort, many…

Software Engineering · Computer Science 2024-09-06 Yacine Majdoub , Eya Ben Charrada

While code review is central to the software development process, it can be tedious and expensive to carry out. In this paper, we investigate whether and how Large Language Models (LLMs) can aid with code reviews. Our investigation focuses…

Software Engineering · Computer Science 2024-03-14 Rasmus Ingemann Tuffveson Jensen , Vali Tawosi , Salwa Alamir

Software vulnerabilities pose significant security challenges and potential risks to society, necessitating extensive efforts in automated vulnerability detection. There are two popular lines of work to address automated vulnerability…

Software Engineering · Computer Science 2024-07-24 Xin Zhou , Duc-Manh Tran , Thanh Le-Cong , Ting Zhang , Ivana Clairine Irsan , Joshua Sumarlin , Bach Le , David Lo

Background: The C and C++ languages hold significant importance in Software Engineering research because of their widespread use in practice. Numerous studies have utilized Machine Learning (ML) and Deep Learning (DL) techniques to detect…

Software Engineering · Computer Science 2024-08-06 Anh The Nguyen , Triet Huynh Minh Le , M. Ali Babar

Background: Leaking sensitive information - such as API keys, tokens, and credentials - in source code remains a persistent security threat. Traditional regex and entropy-based tools often generate high false positives due to limited…

Software Engineering · Computer Science 2025-07-29 Md Nafiu Rahman , Sadif Ahmed , Zahin Wahab , S M Sohan , Rifat Shahriyar

The prevalence of malicious packages in open-source repositories, such as PyPI, poses a critical threat to the software supply chain. While Large Language Models (LLMs) have emerged as a promising tool for automated security tasks, their…

Cryptography and Security · Computer Science 2026-03-03 Ahmed Ryan , Ibrahim Khalil , Abdullah Al Jahid , Md Erfan , Sungbin Park , Akond Ashfaque Ur Rahman , Md Rayhanur Rahman

The rapid advancement of large language models (LLMs) such as GPT-4 has revolutionized the landscape of software engineering, positioning these models at the core of modern development practices. As we anticipate these models to evolve into…

Software Engineering · Computer Science 2025-06-16 Jianian Gong , Nachuan Duan , Ziheng Tao , Zhaohui Gong , Yuan Yuan , Minlie Huang

Large Language Models (LLMs) have demonstrated remarkable capabilities in code generation, but their proficiency in producing secure code remains a critical, under-explored area. Existing benchmarks often fall short by relying on synthetic…

Cryptography and Security · Computer Science 2026-02-02 Yanlin Wang , Ziyao Zhang , Chong Wang , Xinyi Xu , Mingwei Liu , Yong Wang , Jiachi Chen , Zibin Zheng

Various Deep Learning-based approaches with pre-trained language models have been proposed for automatically repairing software vulnerabilities. However, these approaches are limited to a specific programming language (C/C++). Recent…

Software Engineering · Computer Science 2025-08-06 Dong wang , Junji Yu , Honglin Shu , Michael Fu , Chakkrit Tantithamthavorn , Yasutaka Kamei , Junjie Chen

In this paper, we present the first comprehensive empirical study of specialized LLM-based detectors and compare them with traditional static analyzers at the project scale. Specifically, our study evaluates five latest and representative…

Software Engineering · Computer Science 2026-01-28 Fengjie Li , Jiajun Jiang , Dongchi Chen , Yingfei Xiong

Large Language Models are a promising tool for automated vulnerability detection, thanks to their success in code generation and repair. However, despite widespread adoption, a critical question remains: Are LLMs truly effective at…

Cryptography and Security · Computer Science 2025-04-21 Yue Li , Xiao Li , Hao Wu , Minghui Xu , Yue Zhang , Xiuzhen Cheng , Fengyuan Xu , Sheng Zhong

Code translation is crucial for cross-language codebase migration, and large language models (LLMs) have emerged as a promising technique to automate this process. However, the security implications of using LLMs for code translation remain…

Cryptography and Security · Computer Science 2025-12-09 Hailong Chang , Guozhu Meng , Shuhui Xiao , Kai Chen , Kun Sun , Yilin Li

Security vulnerabilities present in a code that has been written in diverse programming languages are among the most critical yet complicated aspects of source code to detect. Static analysis tools based on rule-based patterns usually do…

Cryptography and Security · Computer Science 2025-08-19 Hael Abdulhakim Ali Humran , Ferdi Sonmez

Code Pre-trained Models (CodePTMs) based vulnerability detection have achieved promising results over recent years. However, these models struggle to generalize as they typically learn superficial mapping from source code to labels instead…

Cryptography and Security · Computer Science 2024-06-07 Xiaohu Du , Ming Wen , Jiahao Zhu , Zifan Xie , Bin Ji , Huijun Liu , Xuanhua Shi , Hai Jin

Large Language Models (LLMs) have emerged as promising tools for malware detection by analyzing code semantics, identifying vulnerabilities, and adapting to evolving threats. However, their reliability under adversarial compiler-level…

Cryptography and Security · Computer Science 2025-09-23 Ekin Böke , Simon Torka

Code smells are symptoms of potential code quality problems that may affect software maintainability, thus increasing development costs and impacting software reliability. Large language models (LLMs) have shown remarkable capabilities for…

Software Engineering · Computer Science 2026-01-16 Saymon Souza , Amanda Santana , Eduardo Figueiredo , Igor Muzetti , João Eduardo Montandon , Lionel Briand

Large Language Models (LLMs) are increasingly being studied for Software Vulnerability Detection (SVD) and Repair (SVR). Individual LLMs have demonstrated code understanding abilities, but they frequently struggle when identifying complex…

Software Engineering · Computer Science 2025-12-16 Arastoo Zibaeirad , Marco Vieira

The significant increase in software production, driven by the acceleration of development cycles over the past two decades, has led to a steady rise in software vulnerabilities, as shown by statistics published yearly by the CVE program.…

Software Engineering · Computer Science 2025-12-11 Dyna Soumhane Ouchebara , Stéphane Dupont

Large language models (LLMs) can detect software vulnerabilities, but how do they actually identify vulnerable code? We address this question using mechanistic interpretability; analyzing the internal computations of a neural network to…

Cryptography and Security · Computer Science 2026-05-29 Syafiq Al Atiiq , Chun Zhou , Christian Gehrmann