Recent work has demonstrated that natural language processing techniques can support consumer protection by automatically detecting unfair clauses in the Terms of Service (ToS) Agreement. This work demonstrates that transformer-based ToS analysis systems are vulnerable to adversarial attacks. We conduct experiments attacking an unfair-clause detector with universal adversarial triggers. Experiments show that a minor perturbation of the text can considerably reduce the detection performance. Moreover, to measure the detectability of the triggers, we conduct a detailed human evaluation study by collecting both answer accuracy and response time from the participants. The results show that the naturalness of the triggers remains key to tricking readers.
Cite
@article{arxiv.2211.15556,
title = {Attack on Unfair ToS Clause Detection: A Case Study using Universal Adversarial Triggers},
author = {Shanshan Xu and Irina Broda and Rashid Haddad and Marco Negrini and Matthias Grabmair},
journal= {arXiv preprint arXiv:2211.15556},
year = {2022}
}