English

A Different Level Text Protection Mechanism With Differential Privacy

Computation and Language 2024-09-06 v1 Artificial Intelligence
Authors: Qingwen Fu
View on arXiv ↗ PDF ↗

Abstract

The article introduces a method for extracting words of different degrees of importance based on the BERT pre-training model and proves the effectiveness of this method. The article also discusses the impact of maintaining the same perturbation results for words of different importance on the overall text utility. This method can be applied to long text protection.

Keywords

bert pre-trained language model text classification

Cite

@article{arxiv.2409.03707,
  title  = {A Different Level Text Protection Mechanism With Differential Privacy},
  author = {Qingwen Fu},
  journal= {arXiv preprint arXiv:2409.03707},
  year   = {2024}
}

Related papers

View all related →
Computation and Language · Computer Science
Disentangling the Linguistic Competence of Privacy-Preserving BERT
Stefan Arnold, Nils Kemmerzell, Annika Schreiner
2023-10-18
Computation and Language · Computer Science
Natural Language Understanding with Privacy-Preserving BERT
Chen Qu, Weize Kong, Liu Yang, Mingyang Zhang +2
2021-08-23
Cryptography and Security · Computer Science
The Limits of Word Level Differential Privacy
Justus Mattern, Benjamin Weggenmann, Florian Kerschbaum
2022-05-05
Computation and Language · Computer Science
Characterizing Stereotypical Bias from Privacy-preserving Pre-Training
Stefan Arnold, Rene Gröbner, Annika Schreiner
2024-07-02
Computation and Language · Computer Science
Differential Privacy for Text Analytics via Natural Text Sanitization
Xiang Yue, Minxin Du, Tianhao Wang, Yaliang Li +2
2021-06-03
Computation and Language · Computer Science
Memorization of Named Entities in Fine-tuned BERT Models
Andor Diera, Nicolas Lell, Aygul Garifullina, Ansgar Scherp
2024-12-02
Computation and Language · Computer Science
Provably Confidential Language Modelling
Xuandong Zhao, Lei Li, Yu-Xiang Wang
2022-06-27
Computation and Language · Computer Science
A Comparative Analysis of Word-Level Metric Differential Privacy: Benchmarking The Privacy-Utility Trade-off
Stephen Meisenbacher, Nihildev Nandakumar, Alexandra Klymenko, Florian Matthes
2024-04-05
Computation and Language · Computer Science
Breaking Down Word Semantics from Pre-trained Language Models through Layer-wise Dimension Selection
Nayoung Choi
2023-10-10
Computation and Language · Computer Science
You Are What You Write: Preserving Privacy in the Era of Large Language Models
Richard Plant, Valerio Giuffrida, Dimitra Gkatzia
2022-04-21
Computation and Language · Computer Science
Differentially Private Decoding in Large Language Models
Jimit Majmudar, Christophe Dupuy, Charith Peris, Sami Smaili +2
2022-09-12
Computation and Language · Computer Science
How to Fine-Tune BERT for Text Classification?
Chi Sun, Xipeng Qiu, Yige Xu, Xuanjing Huang
2020-02-06
Computation and Language · Computer Science
1-Diffractor: Efficient and Utility-Preserving Text Obfuscation Leveraging Word-Level Metric Differential Privacy
Stephen Meisenbacher, Maulik Chevli, Florian Matthes
2024-05-06
Information Retrieval · Computer Science
The Power of Selecting Key Blocks with Local Pre-ranking for Long Document Information Retrieval
Minghan Li, Diana Nicoleta Popa, Johan Chagnon, Yagmur Gizem Cinar +1
2022-10-18
Computation and Language · Computer Science
Using Selective Masking as a Bridge between Pre-training and Fine-tuning
Tanish Lad, Himanshu Maheshwari, Shreyas Kottukkal, Radhika Mamidi
2022-11-28
Machine Learning · Computer Science
Differentially Private Language Models Benefit from Public Pre-training
Gavin Kerrigan, Dylan Slack, Jens Tuyls
2020-10-27
Computation and Language · Computer Science
Evaluating BERT-based Pre-training Language Models for Detecting Misinformation
Rini Anggrainingsih, Ghulam Mubashar Hassan, Amitava Datta
2022-03-16
Computation and Language · Computer Science
A Collocation-based Method for Addressing Challenges in Word-level Metric Differential Privacy
Stephen Meisenbacher, Maulik Chevli, Florian Matthes
2024-07-02
Computation and Language · Computer Science
Differentially Private Distributed Learning for Language Modeling Tasks
Vadim Popov, Mikhail Kudinov, Irina Piontkovskaya, Petr Vytovtov +1
2018-03-07
Computation and Language · Computer Science
Bag of Tricks for Training Data Extraction from Language Models
Weichen Yu, Tianyu Pang, Qian Liu, Chao Du +4
2023-06-02
Machine Learning · Computer Science
Privacy Preserving In-Context-Learning Framework for Large Language Models
Bishnu Bhusal, Manoj Acharya, Ramneet Kaur, Colin Samplawski +4
2025-11-20
Cryptography and Security · Computer Science
Generalised Differential Privacy for Text Document Processing
Natasha Fernandes, Mark Dras, Annabelle McIver
2019-02-06
Machine Learning · Computer Science
Differentially Private Model Compression
Fatemehsadat Mireshghallah, Arturs Backurs, Huseyin A Inan, Lukas Wutschitz +1
2022-06-07
Computation and Language · Computer Science
Planting and Mitigating Memorized Content in Predictive-Text Language Models
C. M. Downey, Wei Dai, Huseyin A. Inan, Kim Laine +2
2022-12-19
Cryptography and Security · Computer Science
ADePT: Auto-encoder based Differentially Private Text Transformation
Satyapriya Krishna, Rahul Gupta, Christophe Dupuy
2021-02-03
R2 v1 2026-06-28T18:35:36.729Z