English
Related papers

Related papers: LLM Cyber Evaluations Don't Capture Real-World Ris…

200 papers

Recent advancements in Large Language Models (LLMs) have significantly enhanced interactions between users and models. These advancements concurrently underscore the need for rigorous safety evaluations due to the manifestation of social…

Computation and Language · Computer Science 2025-03-26 Dahyun Jung , Seungyoon Lee , Hyeonseok Moon , Chanjun Park , Heuiseok Lim

Large language models (LLMs) have revolutionized how we interact with machines. However, this technological advancement has been paralleled by the emergence of "Mallas," malicious services operating underground that exploit LLMs for…

Computation and Language · Computer Science 2024-08-12 Garrett Crumrine , Izzat Alsmadi , Jesus Guerrero , Yuvaraj Munian

While automated vulnerability detection techniques have made promising progress in detecting security vulnerabilities, their scalability and applicability remain challenging. The remarkable performance of Large Language Models (LLMs), such…

Cryptography and Security · Computer Science 2024-10-24 Avishree Khare , Saikat Dutta , Ziyang Li , Alaia Solko-Breslin , Rajeev Alur , Mayur Naik

This study investigates how access to Large Language Models (LLMs) and varying levels of professional software development experience affect the prioritization of cybersecurity requirements for web applications. Twenty-three postgraduate…

Software Engineering · Computer Science 2025-10-07 Damjan Fujs , Damjan Vavpotič , Tomaž Hovelja , Marko Poženel

Large Language Models (LLMs) are increasingly employed for simulating human behaviors across diverse domains. However, our position is that current LLM-based human simulations remain insufficiently reliable, as evidenced by significant…

Computation and Language · Computer Science 2025-12-02 Qian Wang , Jiaying Wu , Zichen Jiang , Zhenheng Tang , Bingqiao Luo , Nuo Chen , Wei Chen , Bingsheng He

The prospect of artificial intelligence (AI) competing in the adversarial landscape of cyber security has long been considered one of the most impactful, challenging, and potentially dangerous applications of AI. Here, we demonstrate a new…

Cryptography and Security · Computer Science 2025-02-25 Michael Kouremetis , Marissa Dotter , Alex Byrne , Dan Martin , Ethan Michalak , Gianpaolo Russo , Michael Threet , Guido Zarrella

The rapid deployment of Large Language Models (LLMs) requires careful consideration of their effect on cybersecurity. Our work aims to improve the selection process of LLMs that are suitable for facilitating Secure Coding (SC). This raises…

Cryptography and Security · Computer Science 2024-08-30 Anton Rydén , Erik Näslund , Elad Michael Schiller , Magnus Almgren

Cyber Ranges (CRs) have emerged as prominent platforms for cybersecurity training and education, especially for Critical Infrastructure (CI) sectors that face rising cyber threats. One way to address these threats is through hands-on…

Cryptography and Security · Computer Science 2025-12-12 Vyron Kampourakis , Georgios Kavallieratos , Georgios Spathoulas , Vasileios Gkioulos , Sokratis Katsikas

The use of Large Language Models (LLMs) in police operations is growing, yet an evaluation framework tailored to police operations remains absent. While LLM's responses may not always be legally incorrect, their unverified use still can…

Computation and Language · Computer Science 2026-01-08 Sangyub Lee , Heedou Kim , Hyeoncheol Kim

Large language models (LLMs) have attracted significant attention for potential applications in digital health, while their application in mental health is subject to ongoing debate. This systematic review aims to evaluate the usage of LLMs…

Computers and Society · Computer Science 2024-08-14 Zhijun Guo , Alvina Lai , Johan Hilge Thygesen , Joseph Farrington , Thomas Keen , Kezhi Li

Applying large language models (LLMs) to modern power systems presents a promising avenue for enhancing decision-making and operational efficiency. However, this action may also incur potential security threats, which have not been fully…

Artificial Intelligence · Computer Science 2024-01-25 Jiaqi Ruan , Gaoqi Liang , Huan Zhao , Guolong Liu , Xianzhuo Sun , Jing Qiu , Zhao Xu , Fushuan Wen , Zhao Yang Dong

Large Language Models (LLMs) have emerged as powerful tools across various domains within cyber security. Notably, recent studies are increasingly exploring LLMs applied to the context of blockchain security (BS). However, there remains a…

Cryptography and Security · Computer Science 2025-03-25 Zheyuan He , Zihao Li , Sen Yang , He Ye , Ao Qiao , Xiaosong Zhang , Xiapu Luo , Ting Chen

With the continuous development of large language models (LLMs), transformer-based models have made groundbreaking advances in numerous natural language processing (NLP) tasks, leading to the emergence of a series of agents that use LLMs as…

Artificial Intelligence · Computer Science 2024-11-15 Yuyou Gan , Yong Yang , Zhe Ma , Ping He , Rui Zeng , Yiming Wang , Qingming Li , Chunyi Zhou , Songze Li , Ting Wang , Yunjun Gao , Yingcai Wu , Shouling Ji

Large language models (LLMs) pose significant risks due to the potential for generating harmful content or users attempting to evade guardrails. Existing studies have developed LLM-based guard models designed to moderate the input and…

Cryptography and Security · Computer Science 2025-02-25 Hongfu Liu , Hengguan Huang , Xiangming Gu , Hao Wang , Ye Wang

With the rapid popularity of large language models such as ChatGPT and GPT-4, a growing amount of attention is paid to their safety concerns. These models may generate insulting and discriminatory content, reflect incorrect social values,…

Computation and Language · Computer Science 2023-04-21 Hao Sun , Zhexin Zhang , Jiawen Deng , Jiale Cheng , Minlie Huang

Large Language Models (LLMs) have shown powerful performance and development prospects and are widely deployed in the real world. However, LLMs can capture social biases from unprocessed training data and propagate the biases to downstream…

Computation and Language · Computer Science 2024-02-22 Yingji Li , Mengnan Du , Rui Song , Xin Wang , Ying Wang

Prompt injection attacks exploit vulnerabilities in large language models (LLMs) to manipulate the model into unintended actions or generate malicious content. As LLM integrated applications gain wider adoption, they face growing…

Cryptography and Security · Computer Science 2024-01-03 Daniel Wankit Yip , Aysan Esmradi , Chun Fai Chan

Large language models (LLMs) have become increasingly integrated with various applications. To ensure that LLMs do not generate unsafe responses, they are aligned with safeguards that specify what content is restricted. However, such…

Computation and Language · Computer Science 2024-05-08 Hongyu Cai , Arjun Arunasalam , Leo Y. Lin , Antonio Bianchi , Z. Berkay Celik

As large language models (LLMs) evolve into autonomous "AI scientists," they promise transformative advances but introduce novel vulnerabilities, from potential "biosafety risks" to "dangerous explosions." Ensuring trustworthy deployment in…

Cryptography and Security · Computer Science 2026-03-20 Saket Sanjeev Chaturvedi , Joshua Bergerson , Tanwi Mallick

Large language models (LLMs) have demonstrated immense utility across various industries. However, as LLMs advance, the risk of harmful outputs increases due to incorrect or malicious instruction prompts. While current methods effectively…

Computation and Language · Computer Science 2025-06-19 Xinyi Zeng , Yuying Shang , Jiawei Chen , Jingyuan Zhang , Yu Tian