English

Can large language models generate salient negative statements?

Computation and Language 2023-09-22 v2 Artificial Intelligence

Abstract

We examine the ability of large language models (LLMs) to generate salient (interesting) negative statements about real-world entities; an emerging research topic of the last few years. We probe the LLMs using zero- and k-shot unconstrained probes, and compare with traditional methods for negation generation, i.e., pattern-based textual extractions and knowledge-graph-based inferences, as well as crowdsourced gold statements. We measure the correctness and salience of the generated lists about subjects from different domains. Our evaluation shows that guided probes do in fact improve the quality of generated negatives, compared to the zero-shot variant. Nevertheless, using both prompts, LLMs still struggle with the notion of factuality of negatives, frequently generating many ambiguous statements, or statements with negative keywords but a positive meaning.

Keywords

Cite

@article{arxiv.2305.16755,
  title  = {Can large language models generate salient negative statements?},
  author = {Hiba Arnaout and Simon Razniewski},
  journal= {arXiv preprint arXiv:2305.16755},
  year   = {2023}
}

Comments

For data, see https://www.mpi-inf.mpg.de/fileadmin/inf/d5/research/negation_in_KBs/data.csv

R2 v1 2026-06-28T10:47:18.910Z