English
Related papers

Related papers: Privacy Collapse: Benign Fine-Tuning Can Break Con…

200 papers

Large language models (LLMs) are excellent few-shot learners. They can perform a wide variety of tasks purely based on natural language prompts provided to them. These prompts contain data of a specific downstream task -- often the private…

Machine Learning · Computer Science 2024-11-19 Haonan Duan , Adam Dziedzic , Mohammad Yaghini , Nicolas Papernot , Franziska Boenisch

Language models, characterized by their black-box nature, often hallucinate and display sensitivity to input perturbations, causing concerns about trust. To enhance trust, it is imperative to gain a comprehensive understanding of the…

Computation and Language · Computer Science 2025-01-03 Vatsal Gupta , Pranshu Pandya , Tushar Kataria , Vivek Gupta , Dan Roth

The remarkable ability of language models (LMs) has also brought challenges at the interface of AI and security. A critical challenge pertains to how much information these models retain and leak about the training data. This is…

Machine Learning · Computer Science 2023-01-31 My H. Dinh , Ferdinando Fioretto

Recent data-extraction attacks have exposed that language models can memorize some training samples verbatim. This is a vulnerability that can compromise the privacy of the model's training data. In this work, we introduce SubMix: a…

Machine Learning · Computer Science 2022-01-05 Antonio Ginart , Laurens van der Maaten , James Zou , Chuan Guo

The privacy concerns associated with the use of Large Language Models (LLMs) have grown recently with the development of LLMs such as ChatGPT. Differential Privacy (DP) techniques are explored in existing work to mitigate their privacy…

Artificial Intelligence · Computer Science 2024-03-08 Tiejin Chen , Longchao Da , Huixue Zhou , Pingzhi Li , Kaixiong Zhou , Tianlong Chen , Hua Wei

Domain fine-tuning is a common path to deploy small instruction-tuned language models as customer-support assistants, yet its effects on safety-aligned behavior and privacy are not well understood. In real deployments, such assistants…

Cryptography and Security · Computer Science 2026-03-03 Jayesh Choudhari , Piyush Kumar Singh

SpeechLLMs are increasingly deployed in professional settings where domain customisation is standard practice: users supply context in prompts with sensitive information, fine-tune on proprietary recordings, or both. We identify and…

Computation and Language · Computer Science 2026-05-28 Maike Züfle , Jan Niehues

Protecting privacy in contemporary NLP models is gaining in importance. So does the need to mitigate social biases of such models. But can we have both at the same time? Existing research suggests that privacy preservation comes at the…

Computation and Language · Computer Science 2023-05-25 Cleo Matzken , Steffen Eger , Ivan Habernal

Instruction tuning has proven effective in enhancing Large Language Models' (LLMs) performance on downstream tasks. However, real-world fine-tuning faces inherent conflicts between model providers' intellectual property protection, clients'…

Machine Learning · Computer Science 2025-01-22 Haonan Shi , Tu Ouyang , An Wang

Fine-tuning-based unlearning methods prevail for preventing targeted harmful, sensitive, or copyrighted information within large language models while preserving overall capabilities. However, the true effectiveness of these methods is…

Computation and Language · Computer Science 2024-10-16 Yihuai Hong , Yuelin Zou , Lijie Hu , Ziqian Zeng , Di Wang , Haiqin Yang

Fine-tuning is a prominent technique to adapt a pre-trained language model to downstream scenarios. In parameter-efficient fine-tuning, only a small subset of modules are trained over the downstream datasets, while leaving the rest of the…

Computation and Language · Computer Science 2023-12-27 Xicong Shen , Yang Liu , Huiqi Liu , Jue Hong , Bing Duan , Zirui Huang , Yunlong Mao , Ye Wu , Di Wu

Large Language Models (LLMs), which bridge the gap between human language understanding and complex problem-solving, achieve state-of-the-art performance on several NLP tasks, particularly in few-shot and zero-shot settings. Despite the…

Cryptography and Security · Computer Science 2025-01-07 Shuai Zhao , Meihuizi Jia , Zhongliang Guo , Leilei Gan , Xiaoyu Xu , Xiaobao Wu , Jie Fu , Yichao Feng , Fengjun Pan , Luu Anh Tuan

As the capabilities of pre-trained large language models (LLMs) continue to advance, the "pre-train and fine-tune" paradigm has become increasingly mainstream, leading to the development of various fine-tuning methods. However, the privacy…

Computation and Language · Computer Science 2025-07-02 Jie Hou , Chuxiong Wu , Lannan Luo , Qiang Zeng

The proliferation of visual sensors in smart home environments, particularly through wearable devices like smart glasses, introduces profound privacy challenges. Existing privacy controls are often static and coarse-grained, failing to…

Human-Computer Interaction · Computer Science 2025-08-04 Shuning Zhang , Ying Ma , Xin Yi , Hewu Li

Current LLM-based services typically require users to submit raw text regardless of its sensitivity. While intuitive, such practice introduces substantial privacy risks, as unauthorized access may expose personal, medical, or legal…

Cryptography and Security · Computer Science 2026-04-09 Jeongho Yoon , Chanhee Park , Yongchan Chun , Hyeonseok Moon , Heuiseok Lim

The advancement of large language models (LLMs) has significantly enhanced the ability to effectively tackle various downstream NLP tasks and unify these tasks into generative pipelines. On the one hand, powerful language models, trained on…

Computation and Language · Computer Science 2024-10-01 Haoran Li , Yulin Chen , Jinglong Luo , Jiecong Wang , Hao Peng , Yan Kang , Xiaojin Zhang , Qi Hu , Chunkit Chan , Zenglin Xu , Bryan Hooi , Yangqiu Song

Large language models (LLMs), renowned for their impressive capabilities in various tasks, have significantly advanced artificial intelligence. Yet, these advancements have raised growing concerns about privacy and security implications. To…

Artificial Intelligence · Computer Science 2024-03-28 Yuqi Yang , Xiaowen Huang , Jitao Sang

The rapid advancements of large language models (LLMs) have raised public concerns about the privacy leakage of personally identifiable information (PII) within their extensive training datasets. Recent studies have demonstrated that an…

Cryptography and Security · Computer Science 2024-08-01 Xiaoyi Chen , Siyuan Tang , Rui Zhu , Shijun Yan , Lei Jin , Zihao Wang , Liya Su , Zhikun Zhang , XiaoFeng Wang , Haixu Tang

Finetuning provides a scalable and cost-effective means of customizing language models for specific tasks or response styles, with greater reliability than prompting or in-context learning. In contrast, the conventional wisdom is that…

Computation and Language · Computer Science 2025-03-11 Eric Zhao , Pranjal Awasthi , Nika Haghtalab

With the rapid development of artificial intelligence, large language models (LLMs) have made remarkable advancements in natural language processing. These models are trained on vast datasets to exhibit powerful language understanding and…

Cryptography and Security · Computer Science 2025-09-22 Shang Wang , Tianqing Zhu , Bo Liu , Ming Ding , Dayong Ye , Wanlei Zhou , Philip S. Yu