English
Related papers

Related papers: Traces of Memorisation in Large Language Models fo…

200 papers

Large Language Models (LLMs) have recently been widely used for code generation. Due to the complexity and opacity of LLMs, little is known about how these models generate code. We made the first attempt to bridge this knowledge gap by…

Software Engineering · Computer Science 2024-05-24 Bonan Kou , Shengmai Chen , Zhijie Wang , Lei Ma , Tianyi Zhang

Studying data memorization in neural language models helps us understand the risks (e.g., to privacy or copyright) associated with models regurgitating training data and aids in the development of countermeasures. Many prior works -- and…

The text generated by large language models is commonly controlled by prompting, where a prompt prepended to a user's query guides the model's output. The prompts used by companies to guide their models are often treated as secrets, to be…

Computation and Language · Computer Science 2024-08-09 Yiming Zhang , Nicholas Carlini , Daphne Ippolito

Large Language Models (LLMs), now a foundation in advancing natural language processing, power applications such as text generation, machine translation, and conversational systems. Despite their transformative potential, these models…

Cryptography and Security · Computer Science 2025-08-05 Kang Chen , Xiuze Zhou , Yuanguo Lin , Jinhe Su , Yuanhui Yu , Li Shen , Fan Lin

Memorization in Large Language Models (LLMs) poses privacy and security risks, as models may unintentionally reproduce sensitive or copyrighted data. Existing analyses focus on average-case scenarios, often neglecting the highly skewed…

Artificial Intelligence · Computer Science 2025-02-04 Hao Li , Di Huang , Ziyu Wang , Amir M. Rahmani

Transformer-based language models for code have shown remarkable performance in various software analytics tasks, but their adoption is hindered by high computational costs, slow inference speeds, and substantial environmental impact. Model…

Software Engineering · Computer Science 2026-04-15 Md. Abdul Awal , Mrigank Rochan , Chanchal K. Roy

In the rapidly evolving domain of artificial intelligence, safeguarding the intellectual property of Large Language Models (LLMs) is increasingly crucial. Current watermarking techniques against model extraction attacks, which rely on…

Cryptography and Security · Computer Science 2024-05-03 Minhao Bai , Kaiyi Pang , Yongfeng Huang

Language models for code (CodeLMs) have emerged as powerful tools for code-related tasks, outperforming traditional methods and standard machine learning approaches. However, these models are susceptible to security vulnerabilities, drawing…

Software Engineering · Computer Science 2025-05-20 Yuchen Chen , Weisong Sun , Chunrong Fang , Zhenpeng Chen , Yifei Ge , Tingxu Han , Quanjun Zhang , Yang Liu , Zhenyu Chen , Baowen Xu

Scientists across disciplines write code for critical activities like data collection and generation, statistical modeling, and visualization. As large language models that can generate code have become widely available, scientists may…

Software Engineering · Computer Science 2025-02-25 Gabrielle O'Brien

We propose a new method for estimating how much a model knows about a datapoint and use it to measure the capacity of modern language models. Prior studies of language model memorization have struggled to disentangle memorization from…

Sequence models, such as Large Language Models (LLMs) and autoregressive image generators, have a tendency to memorize and inadvertently leak sensitive information. While this tendency has critical legal implications, existing tools are…

Cryptography and Security · Computer Science 2025-06-06 Lorenzo Rossi , Michael Aerni , Jie Zhang , Florian Tramèr

Data contamination in model evaluation has become increasingly prevalent with the growing popularity of large language models. It allows models to "cheat" via memorisation instead of displaying true capabilities. Therefore, contamination…

Computation and Language · Computer Science 2024-01-30 Yucheng Li , Frank Guerin , Chenghua Lin

Recent research demonstrated that training large language models involves memorization of a significant fraction of training data. Such memorization can lead to privacy violations when training on sensitive user data and thus motivates the…

Machine Learning · Computer Science 2025-10-29 Vitaly Feldman , Guy Kornowski , Xin Lyu

Large Language Models (LLMs) memorize, and thus, among huge amounts of uncontrolled data, may memorize Personally Identifiable Information (PII), which should not be stored and, consequently, not leaked. In this paper, we introduce Private…

Cryptography and Security · Computer Science 2025-08-22 Elena Sofia Ruzzetti , Giancarlo A. Xompero , Davide Venditti , Fabio Massimo Zanzotto

Understanding whether and to what extent large language models (LLMs) have memorised training data has important implications for the reliability of their output and the privacy of their training data. In order to cleanly measure and…

Computation and Language · Computer Science 2024-07-30 Till Speicher , Mohammad Aflah Khan , Qinyuan Wu , Vedant Nanda , Soumi Das , Bishwamittra Ghosh , Krishna P. Gummadi , Evimaria Terzi

Despite various approaches being employed to detect vulnerabilities, the number of reported vulnerabilities shows an upward trend over the years. This suggests the problems are not caught before the code is released, which could be caused…

Cryptography and Security · Computer Science 2025-02-14 Karl Tamberg , Hayretdin Bahsi

While automated vulnerability detection techniques have made promising progress in detecting security vulnerabilities, their scalability and applicability remain challenging. The remarkable performance of Large Language Models (LLMs), such…

Cryptography and Security · Computer Science 2024-10-24 Avishree Khare , Saikat Dutta , Ziyang Li , Alaia Solko-Breslin , Rajeev Alur , Mayur Naik

Large Language Models (LLMs) are huge artificial neural networks which primarily serve to generate text, but also provide a very sophisticated probabilistic model of language use. Since generating a semantically consistent text requires a…

Computation and Language · Computer Science 2024-04-09 Romuald A. Janik

Memorization in large language models poses critical risks for privacy and fairness as these systems scale to billions of parameters. While previous studies established correlations between memorization and factors like token frequency and…

Machine Learning · Computer Science 2025-09-01 Jie Zhang , Qinghua Zhao , Chi-ho Lin , Zhongfeng Kang , Lei Li

This paper explores the risk that a large language model (LLM) trained for code generation on data mined from software repositories will generate content that discloses sensitive information included in its training data. We decompose this…

Cryptography and Security · Computer Science 2026-04-16 Rafiqul Rabin , Sean McGregor , Nick Judd