English

Span-Level Hallucination Detection for LLM-Generated Answers

Computation and Language 2025-04-29 v1

Abstract

Detecting spans of hallucination in LLM-generated answers is crucial for improving factual consistency. This paper presents a span-level hallucination detection framework for the SemEval-2025 Shared Task, focusing on English and Arabic texts. Our approach integrates Semantic Role Labeling (SRL) to decompose the answer into atomic roles, which are then compared with a retrieved reference context obtained via question-based LLM prompting. Using a DeBERTa-based textual entailment model, we evaluate each role semantic alignment with the retrieved context. The entailment scores are further refined through token-level confidence measures derived from output logits, and the combined scores are used to detect hallucinated spans. Experiments on the Mu-SHROOM dataset demonstrate competitive performance. Additionally, hallucinated spans have been verified through fact-checking by prompting GPT-4 and LLaMA. Our findings contribute to improving hallucination detection in LLM-generated responses.

Keywords

Cite

@article{arxiv.2504.18639,
  title  = {Span-Level Hallucination Detection for LLM-Generated Answers},
  author = {Passant Elchafei and Mervet Abu-Elkheir},
  journal= {arXiv preprint arXiv:2504.18639},
  year   = {2025}
}
R2 v1 2026-06-28T23:11:52.535Z