English
Related papers

Related papers: Top Score on the Wrong Exam: On Benchmarking in Ma…

200 papers

Malware, or software designed with harmful intent, is an ever-evolving threat that can have drastic effects on both individuals and institutions. Neural network malware classification systems are key tools for combating these threats but…

Cryptography and Security · Computer Science 2024-04-09 Preston K. Robinette , Diego Manzanas Lopez , Serena Serbinowska , Kevin Leach , Taylor T. Johnson

Large Language Models (LLMs), characterized by being trained on broad amounts of data in a self-supervised manner, have shown impressive performance across a wide range of tasks. Indeed, their generative abilities have aroused interest on…

Machine Learning · Computer Science 2024-07-30 Jorge García-Carrasco , Alejandro Maté , Juan Trujillo

Software vulnerabilities (SVs) have emerged as a prevalent and critical concern for safety-critical security systems. This has spurred significant advancements in utilizing AI-based methods, including machine learning and deep learning, for…

Software Engineering · Computer Science 2025-10-07 Van Nguyen , Surya Nepal , Tingmin Wu , Xingliang Yuan , Carsten Rudolph

This paper proposes a pipeline for quantitatively evaluating interactive Large Language Models (LLMs) using publicly available datasets. We carry out an extensive technical evaluation of LLMs using Big-Vul covering four different common…

Software Engineering · Computer Science 2024-07-09 Xin Yin , Chao Ni , Shaohua Wang

While much of the current research in deep learning-based vulnerability detection relies on disassembled binaries, this paper explores the feasibility of extracting features directly from raw x86-64 machine code. Although assembly language…

Cryptography and Security · Computer Science 2026-01-15 Mitchell Petingola

Deep learning (DL) models have become increasingly popular in identifying software vulnerabilities. Prior studies found that vulnerabilities across different vulnerable programs may exhibit similar vulnerable scopes, implicitly forming…

Cryptography and Security · Computer Science 2023-06-13 Michael Fu , Trung Le , Van Nguyen , Chakkrit Tantithamthavorn , Dinh Phung

The pursuit of leaderboard rankings in Large Language Models (LLMs) has created a fundamental paradox: models excel at standardized tests while failing to demonstrate genuine language understanding and adaptability. Our systematic analysis…

Computation and Language · Computer Science 2024-12-06 Sourav Banerjee , Ayushi Agarwal , Eishkaran Singh

Machine Learning (ML) has become a ubiquitous tool for predicting and classifying data and has found application in several problem domains, including Software Development (SD). This paper reviews the literature between 2000 and 2019 on the…

Vulnerability detection is a crucial yet challenging technique for ensuring the security of software systems. Currently, most deep learning-based vulnerability detection methods focus on stand-alone functions, neglecting the complex…

Software Engineering · Computer Science 2025-06-27 Shaojian Qiu , Mengyang Huang , Jiahao Cheng

Recent advancements in large language models (LLMs) have highlighted the potential for vulnerability detection, a crucial component of software quality assurance. Despite this progress, most studies have been limited to the perspective of a…

Software Engineering · Computer Science 2024-05-21 Zhenyu Mao , Jialong Li , Dongming Jin , Munan Li , Kenji Tei

Software vulnerabilities (SVs) pose a critical threat to safety-critical systems, driving the adoption of AI-based approaches such as machine learning and deep learning for software vulnerability detection. Despite promising results, most…

Cryptography and Security · Computer Science 2025-10-07 Van Nguyen , Surya Nepal , Xingliang Yuan , Tingmin Wu , Fengchao Chen , Carsten Rudolph

While automated vulnerability detection techniques have made promising progress in detecting security vulnerabilities, their scalability and applicability remain challenging. The remarkable performance of Large Language Models (LLMs), such…

Cryptography and Security · Computer Science 2024-10-24 Avishree Khare , Saikat Dutta , Ziyang Li , Alaia Solko-Breslin , Rajeev Alur , Mayur Naik

Software vulnerability detection is generally supported by automated static analysis tools, which have recently been reinforced by deep learning (DL) models. However, despite the superior performance of DL-based approaches over rule-based…

Software Engineering · Computer Science 2024-05-03 Yanjing Yang , Xin Zhou , Runfeng Mao , Jinwei Xu , Lanxin Yang , Yu Zhangm , Haifeng Shen , He Zhang

Language models (LMs) show promise for vulnerability detection but struggle with long, real-world code due to sparse and uncertain vulnerability locations. These issues, exacerbated by token limits, often cause models to miss…

Software Engineering · Computer Science 2025-07-16 Xinran Zheng , Xingzhi Qian , Huichi Zhou , Shuo Yang , Yiling He , Suman Jana , Lorenzo Cavallaro

Software vulnerabilities pose significant risks to the security and integrity of software systems. Although prior studies have explored vulnerability detection using deep learning and pre-trained models, these approaches often fail to…

Software Engineering · Computer Science 2025-09-04 Qiheng Mao , Zhenhao Li , Xing Hu , Kui Liu , Xin Xia , Jianling Sun

The increasing adoption of Large Language Models (LLMs) in software engineering has sparked interest in their use for software vulnerability detection. However, the rapid development of this field has resulted in a fragmented research…

Software Engineering · Computer Science 2025-12-22 Sabrina Kaniewski , Fabian Schmidt , Markus Enzweiler , Michael Menth , Tobias Heer

Fine-grained software vulnerability detection is an important and challenging problem. Ideally, a detection system (or detector) not only should be able to detect whether or not a program contains vulnerabilities, but also should be able to…

Cryptography and Security · Computer Science 2020-01-09 Deqing Zou , Sujuan Wang , Shouhuai Xu , Zhen Li , Hai Jin

Applying security patches in open source software timely is critical for ensuring the security of downstream applications. However, it is challenging to apply these patches promptly because notifications of patches are often incomplete and…

Cryptography and Security · Computer Science 2024-06-11 Tianyu Chen , Lin Li , Taotao Qian , Jingyi Liu , Wei Yang , Ding Li , Guangtai Liang , Qianxiang Wang , Tao Xie

Traditional vulnerability detection methods rely heavily on predefined rule matching, which often fails to capture vulnerabilities accurately. With the rise of large language models (LLMs), leveraging their ability to understand code…

Cryptography and Security · Computer Science 2025-11-26 Xiang Li , Yueci Su , Jiahao Liu , Zhiwei Lin , Yuebing Hou , Peiming Gao , Yuanchao Zhang

Deep learning-based approaches, particularly those leveraging pre-trained language models (PLMs), have shown promise in automated software vulnerability detection. However, existing methods are predominantly limited to specific programming…

Software Engineering · Computer Science 2025-05-13 Junji Yu , Honglin Shu , Michael Fu , Dong Wang , Chakkrit Tantithamthavorn , Yasutaka Kamei , Junjie Chen