English
Related papers

Related papers: Evaluating Large Language Models in Vulnerability …

200 papers

Human developers can produce code with cybersecurity bugs. Can emerging 'smart' code completion tools help repair those bugs? In this work, we examine the use of large language models (LLMs) for code (such as OpenAI's Codex and AI21's…

Cryptography and Security · Computer Science 2022-08-16 Hammond Pearce , Benjamin Tan , Baleegh Ahmad , Ramesh Karri , Brendan Dolan-Gavitt

The use of Large Language Models (LLMs) is proliferating, yet their performance is observed to vary based on prompting styles and tones. In this study, we investigate both whether and how tonal variations in prompts lead to disparate LLM…

Artificial Intelligence · Computer Science 2026-05-29 Om Dobariya , Akhil Kumar

Large Language Models (LLMs) are swiftly advancing in architecture and capability, and as they integrate more deeply into complex systems, the urgency to scrutinize their security properties grows. This paper surveys research in the…

Computation and Language · Computer Science 2023-10-18 Erfan Shayegani , Md Abdullah Al Mamun , Yu Fu , Pedram Zaree , Yue Dong , Nael Abu-Ghazaleh

Large Language Model (LLM)-generated data is increasingly used in software analytics, but it is unclear how this data compares to human-written data, particularly when models are exposed to adversarial scenarios. Adversarial attacks can…

Software Engineering · Computer Science 2025-05-07 Md. Abdul Awal , Mrigank Rochan , Chanchal K. Roy

Large language models (LLMs), especially generative pre-trained transformers (GPTs), have recently demonstrated outstanding ability in information comprehension and problem-solving. This has motivated many studies in applying LLMs to…

Machine Learning · Computer Science 2024-05-21 Han Zhang , Akram Bin Sediq , Ali Afana , Melike Erol-Kantarci

In recent years, JavaScript has become the most widely used programming language, especially in web development. However, writing secure JavaScript code is not trivial, and programmers often make mistakes that lead to security…

Cryptography and Security · Computer Science 2024-03-21 Tan Khang Le , Saba Alimadadi , Steven Y. Ko

Large Language Models (LLMs) are increasingly being studied for Software Vulnerability Detection (SVD) and Repair (SVR). Individual LLMs have demonstrated code understanding abilities, but they frequently struggle when identifying complex…

Software Engineering · Computer Science 2025-12-16 Arastoo Zibaeirad , Marco Vieira

Traditional vulnerability detection methods rely heavily on predefined rule matching, which often fails to capture vulnerabilities accurately. With the rise of large language models (LLMs), leveraging their ability to understand code…

Cryptography and Security · Computer Science 2025-11-26 Xiang Li , Yueci Su , Jiahao Liu , Zhiwei Lin , Yuebing Hou , Peiming Gao , Yuanchao Zhang

Large language models (LLMs) have undergone rapid evolution and achieved remarkable results in recent times. OpenAI's ChatGPT, backed by GPT-3.5 or GPT-4, has gained instant popularity due to its strong capability across a wide range of…

Cryptography and Security · Computer Science 2023-12-12 Fangzhou Wu , Qingzhao Zhang , Ati Priya Bajaj , Tiffany Bao , Ning Zhang , Ruoyu "Fish" Wang , Chaowei Xiao

Code quality is an attribute composed of various metrics, such as complexity, readability, testability, interoperability, reusability, and the use of good or bad practices, among others. Static code analysis tools aim to measure a set of…

Software Engineering · Computer Science 2024-10-07 Igor Regis da Silva Simões , Elaine Venson

Although Large Language Models (LLMs) have demonstrated significant capabilities in executing complex tasks in a zero-shot manner, they are susceptible to jailbreak attacks and can be manipulated to produce harmful outputs. Recently, a…

Cryptography and Security · Computer Science 2024-11-07 Zhao Xu , Fan Liu , Hao Liu

Detecting vulnerabilities in source code remains a critical yet challenging task, especially when benign and vulnerable functions share significant similarities. In this work, we introduce VulTrial, a courtroom-inspired multi-agent…

Understanding code represents a core ability needed for automating software development tasks. While foundation models like LLMs show impressive results across many software engineering challenges, the extent of their true semantic…

Software Engineering · Computer Science 2025-04-16 Serge Lionel Nikiema , Jordan Samhi , Abdoul Kader Kaboré , Jacques Klein , Tegawendé F. Bissyandé

Software vulnerabilities remain a persistent risk, yet static and dynamic analyses often overlook structural dependencies that shape insecure behaviors. Viewing programs as heterogeneous graphs, we capture control- and data-flow relations…

Software Engineering · Computer Science 2025-10-14 Jugal Gajjar , Kaustik Ranaware , Kamalasankari Subramaniakuppusamy

In the current cybersecurity landscape, protecting military devices such as communication and battlefield management systems against sophisticated cyber attacks is crucial. Malware exploits vulnerabilities through stealth methods, often…

Cryptography and Security · Computer Science 2024-05-16 Pedro Miguel Sánchez Sánchez , Alberto Huertas Celdrán , Gérôme Bovet , Gregorio Martínez Pérez

Detecting life-threatening language is essential for safeguarding individuals in distress, promoting mental health and well-being, and preventing potential harm and loss of life. This paper presents an effective approach to identifying…

Computation and Language · Computer Science 2025-06-13 Thanh Thi Nguyen , Campbell Wilson , Janis Dalins

This study systematically analyzes the vulnerability of 36 large language models (LLMs) to various prompt injection attacks, a technique that leverages carefully crafted prompts to elicit malicious LLM behavior. Across 144 prompt injection…

Context. Large Language Models (LLMs) are increasingly embedded in software engineering workflows for tasks including code generation, summarization, repair, and testing. Empirical studies report productivity gains, improved comprehension,…

Software Engineering · Computer Science 2026-01-30 Bruno Claudino Matias , Savio Freire , Juliana Freitas , Felipe Fronchetti , Kostadin Damevski , Rodrigo Spinola

Large Language Models (LLMs) have strong capabilities in code comprehension, but fine-tuning costs and semantic alignment issues limit their project-specific optimization; conversely, code models such CodeBERT are easy to fine-tune, but it…

Software Engineering · Computer Science 2024-07-22 Ziliang Wang , Ge Li , Jia Li , Yingfei Xiong , Jia Li , Meng Yan , Zhi Jin

Large Language Models (LLMs) are powerful tools for modern applications, but their computational demands limit accessibility. Quantization offers efficiency gains, yet its impact on safety and trustworthiness remains poorly understood. To…

Cryptography and Security · Computer Science 2025-07-01 Artyom Kharinaev , Viktor Moskvoretskii , Egor Shvetsov , Kseniia Studenikina , Bykov Mikhail , Evgeny Burnaev