LLMs 提示用于图生成：幻觉与生成能力
摘要
大型语言模型（LLMs）如今被用于各种任务。在本文中，我们研究它们在背诵和生成图表方面的能力。我们首先研究 LLMs 背诵文献中已知图表的能力（例如 Karate club 或图谱 atlas4）。其次，我们质疑 LLMs 的生成能力，通过要求生成 Erdos-Renyi 随机图。与可能记忆其抓取训练集中包含的某些 Erdos-Renyi 图相反，这项第二项调查旨在研究 LLMs 可能的涌现属性。对于这两个任务，我们提出了一种衡量其错误的度量标准，以幻觉（即作为事实返回的错误信息）的视角进行。我们最显著的发现是，图表幻觉的幅度可以 characterize 某些 LLMs 的优势。Indeed, for the recitation task, we observe that graph hallucinations correlate with the Hallucination Leaderboard, a hallucination rank that leverages 10, 000 times more prompts to obtain its ranking. For the generation task, we find surprisingly good and reproducible results in most of LLMs. We believe this to constitute a starting point for more in-depth studies of this emergent capability and a challenging benchmark for their improvements. Altogether, these two aspects of LLMs capabilities bridge a gap between the network science and machine learning communities.
关键词
引用
@article{arxiv.2409.00159,
title = {LLMs Prompted for Graphs: Hallucinations and Generative Capabilities},
author = {Gurvan Richardeau and Samy Chali and Erwan Le Merrer and Camilla Penzo and Gilles Tredan},
journal= {arXiv preprint arXiv:2409.00159},
year = {2025}
}
备注
A preliminary version of this work appeared in the Complex Networks 2024 conference, under the title "LLMs hallucinate graphs too: a structural perspective"