We present two LLM-based approaches to zero-shot source-and-target belief prediction on FactBank: a unified system that identifies events, sources, and belief labels in a single pass, and a hybrid approach that uses a fine-tuned DeBERTa tagger for event detection. We show that multiple open-sourced, closed-source, and reasoning-based LLMs struggle with the task. Using the hybrid approach, we achieve new state-of-the-art results on FactBank and offer a detailed error analysis. Our approach is then tested on the Italian belief corpus ModaFact.
Cite
@article{arxiv.2502.08777,
title = {Zero-Shot Belief: A Hard Problem for LLMs},
author = {John Murzaku and Owen Rambow},
journal= {arXiv preprint arXiv:2502.08777},
year = {2025}
}