English

HalluClear: Diagnosing, Evaluating and Mitigating Hallucinations in GUI Agents

Artificial Intelligence 2026-04-21 v1

Abstract

While progress in GUI agents has been largely driven by industrial-scale training, ungrounded hallucinations often trigger cascading failures in real-world deployments.Unlike general VLM domains, the GUI agent field lacks a hallucination-focused suite for fine-grained diagnosis, reliable evaluation, and targeted mitigation.To bridge this gap, we introduce HalluClear, a comprehensive suite for hallucination mitigation in GUI agents as a complement to computation-intensive scaling. HalluClear comprises: (1) a GUI-specific hallucination taxonomy derived from empirical failure analysis; (2) a calibrated three-stage evaluation workflow which enhances VLM-as-a-judge reliability via expert-annotated benchmarking and ensemble credibility estimation; and (3) a mitigation scheme based on closed-loop structured reasoning, enabling lightweight continual post-training with cold-start initialization for both generalist and GUI-specialist agents. Experiments across representative agents and public benchmarks demonstrate that post-training on only 9K samples within our suite can significantly reduce hallucinations, thereby improving grounding and action fidelity, offering a compute-efficient pathway to robust GUI automation.

Keywords

Cite

@article{arxiv.2604.17284,
  title  = {HalluClear: Diagnosing, Evaluating and Mitigating Hallucinations in GUI Agents},
  author = {Chao Jin and Wenkui Yang and Hao Sun and Yuqi Liao and Qianyi Jiang and Kai Zhou and Jie Cao and Ran He and Huaibo Huang},
  journal= {arXiv preprint arXiv:2604.17284},
  year   = {2026}
}

Comments

47 pages, 44 figures

R2 v1 2026-07-01T12:16:37.311Z