With the widespread adoption of Large Language Models (LLMs), concerns about potential misuse have emerged. To this end, watermarking has been adapted to LLM, enabling a simple and effective way to detect and monitor generated text. However, while the existing methods can differentiate between watermarked and unwatermarked text with high accuracy, they often face a trade-off between the quality of the generated text and the effectiveness of the watermarking process. In this work, we present a novel type of LLM watermark, Sparse Watermark, which aims to mitigate this trade-off by applying watermarks to a small subset of generated tokens distributed across the text. The key strategy involves anchoring watermarked tokens to words that have specific Part-of-Speech (POS) tags. Our experimental results demonstrate that the proposed watermarking scheme achieves high detectability while generating text that outperforms previous LLM watermarking methods in quality across various tasks
@article{arxiv.2407.13803,
title = {Less is More: Sparse Watermarking in LLMs with Enhanced Text Quality},
author = {Duy C. Hoang and Hung T. Q. Le and Rui Chu and Ping Li and Weijie Zhao and Yingjie Lao and Khoa D. Doan},
journal= {arXiv preprint arXiv:2407.13803},
year = {2024}
}