English
Related papers

Related papers: Assessing and Mitigating Data Memorization Risks i…

200 papers

Large language models (LLMs) are sophisticated artificial intelligence systems that enable machines to generate human-like text with remarkable precision. While LLMs offer significant technological progress, their development using vast…

Cryptography and Security · Computer Science 2025-06-23 Yashothara Shanmugarasa , Ming Ding , M. A. P Chamikara , Thierry Rakotoarivelo

The discourse on privacy risks in Large Language Models (LLMs) has disproportionately focused on verbatim memorization of training data, while a constellation of more immediate and scalable privacy threats remain underexplored. This…

Cryptography and Security · Computer Science 2025-10-03 Niloofar Mireshghallah , Tianshi Li

Large language models (LLMs) have demonstrated significant success in various domain-specific tasks, with their performance often improving substantially after fine-tuning. However, fine-tuning with real-world data introduces privacy risks.…

Cryptography and Security · Computer Science 2025-01-30 Atilla Akkus , Masoud Poorghaffar Aghdam , Mingjie Li , Junjie Chu , Michael Backes , Yang Zhang , Sinem Sav

With the rapid adoption of Federated Learning (FL) as the training and tuning protocol for applications utilizing Large Language Models (LLMs), recent research highlights the need for significant modifications to FL to accommodate the…

Cryptography and Security · Computer Science 2024-03-11 Minh N. Vu , Truc Nguyen , Tre' R. Jeter , My T. Thai

Large language models (LLMs) have been proven capable of memorizing their training data, which can be extracted through specifically designed prompts. As the scale of datasets continues to grow, privacy risks arising from memorization have…

Computation and Language · Computer Science 2023-11-07 Zhenhong Zhou , Jiuyang Xiang , Chaomeng Chen , Sen Su

Fine-tuning large language models on private data for downstream applications poses significant privacy risks in potentially exposing sensitive information. Several popular community platforms now offer convenient distribution of a large…

Machine Learning · Computer Science 2024-09-02 Md Rafi Ur Rashid , Jing Liu , Toshiaki Koike-Akino , Shagufta Mehnaz , Ye Wang

This study investigates the mechanisms and factors influencing memorization in fine-tuned large language models (LLMs), with a focus on the medical domain due to its privacy-sensitive nature. We examine how different aspects of the…

Computation and Language · Computer Science 2025-08-06 Danil Savine

Large Language Models (LLMs) are known to memorize significant portions of their training data. Parts of this memorized content have been shown to be extractable by simply querying the model, which poses a privacy risk. We present a novel…

Computation and Language · Computer Science 2023-05-22 Mustafa Safa Ozdayi , Charith Peris , Jack FitzGerald , Christophe Dupuy , Jimit Majmudar , Haidar Khan , Rahil Parikh , Rahul Gupta

With the widespread application of large language models (LLMs), user privacy protection has become a significant research topic. Existing privacy preference modeling methods often rely on large-scale user data, making effective privacy…

Cryptography and Security · Computer Science 2025-05-13 Haowei Yang , Qingyi Lu , Yang Wang , Sibei Liu , Jiayun Zheng , Ao Xiang

Large Language Models (LLMs) trained on massive data capture rich information embedded in the training data. However, this also introduces the risk of privacy leakage, particularly involving personally identifiable information (PII).…

Computation and Language · Computer Science 2025-06-10 Wenshuo Dong , Qingsong Yang , Shu Yang , Lijie Hu , Meng Ding , Wanyu Lin , Tianhang Zheng , Di Wang

Large language models (LLMs) have emerged as powerful tools for tackling complex tasks across diverse domains, but they also raise privacy concerns when fine-tuned on sensitive data due to potential memorization. While differential privacy…

Computation and Language · Computer Science 2024-08-19 Lynn Chua , Badih Ghazi , Yangsibo Huang , Pritish Kamath , Ravi Kumar , Daogao Liu , Pasin Manurangsi , Amer Sinha , Chiyuan Zhang

Fine-tuning is a common and effective method for tailoring large language models (LLMs) to specialized tasks and applications. In this paper, we study the privacy implications of fine-tuning LLMs on user data. To this end, we consider a…

Cryptography and Security · Computer Science 2024-02-27 Nikhil Kandpal , Krishna Pillutla , Alina Oprea , Peter Kairouz , Christopher A. Choquette-Choo , Zheng Xu

Large language models are shown to present privacy risks through memorization of training data, and several recent works have studied such risks for the pre-training phase. Little attention, however, has been given to the fine-tuning phase…

Computation and Language · Computer Science 2022-11-07 Fatemehsadat Mireshghallah , Archit Uniyal , Tianhao Wang , David Evans , Taylor Berg-Kirkpatrick

The proliferation of Large Language Models (LLMs) has driven considerable interest in fine-tuning them with domain-specific data to create specialized language models. Nevertheless, such domain-specific fine-tuning data often contains…

Computation and Language · Computer Science 2024-10-29 Yijia Xiao , Yiqiao Jin , Yushi Bai , Yue Wu , Xianjun Yang , Xiao Luo , Wenchao Yu , Xujiang Zhao , Yanchi Liu , Quanquan Gu , Haifeng Chen , Wei Wang , Wei Cheng

Pretrained Language Models (LMs) memorize a vast amount of knowledge during initial pretraining, including information that may violate the privacy of personal lives and identities. Previous work addressing privacy issues for language…

Computation and Language · Computer Science 2022-12-20 Joel Jang , Dongkeun Yoon , Sohee Yang , Sungmin Cha , Moontae Lee , Lajanugen Logeswaran , Minjoon Seo

Large language models (LLMs) have significantly transformed natural language understanding and generation, but they raise privacy concerns due to potential exposure of sensitive information. Studies have highlighted the risk of information…

Machine Learning · Computer Science 2025-11-20 Bishnu Bhusal , Manoj Acharya , Ramneet Kaur , Colin Samplawski , Anirban Roy , Adam D. Cobb , Rohit Chadha , Susmit Jha

With the rapid development of artificial intelligence, large language models (LLMs) have made remarkable advancements in natural language processing. These models are trained on vast datasets to exhibit powerful language understanding and…

Cryptography and Security · Computer Science 2025-09-22 Shang Wang , Tianqing Zhu , Bo Liu , Ming Ding , Dayong Ye , Wanlei Zhou , Philip S. Yu

Recent large-scale natural language processing (NLP) systems use a pre-trained Large Language Model (LLM) on massive and diverse corpora as a headstart. In practice, the pre-trained model is adapted to a wide array of tasks via fine-tuning…

Computation and Language · Computer Science 2022-09-12 Jimit Majmudar , Christophe Dupuy , Charith Peris , Sami Smaili , Rahul Gupta , Richard Zemel

Large Language Models (LLMs) have demonstrated remarkable capabilities across a wide range of tasks, yet they also exhibit memorization of their training data. This phenomenon raises critical questions about model behavior, privacy risks,…

Machine Learning · Computer Science 2025-12-15 Alexander Xiong , Xuandong Zhao , Aneesh Pappu , Dawn Song

The privacy concerns associated with the use of Large Language Models (LLMs) have grown recently with the development of LLMs such as ChatGPT. Differential Privacy (DP) techniques are explored in existing work to mitigate their privacy…

Artificial Intelligence · Computer Science 2024-03-08 Tiejin Chen , Longchao Da , Huixue Zhou , Pingzhi Li , Kaixiong Zhou , Tianlong Chen , Hua Wei