English

From Intent to Evidence: A Categorical Approach for Structural Evaluation of Deep Research Agents

Machine Learning 2026-04-30 v2

Abstract

Deep Research Agents (DRAs) aim to answer complex questions by searching the web, checking evidence, and synthesizing conclusions across heterogeneous sources. We introduce a category-theoretic framework for evaluating and improving such agents. The framework treats deep research as a structured mapping from user intent to evidence-grounded conclusions, making retrieval traces, cross-source alignment, and final synthesis explicit. Guided by this view, we derive a mechanism-aware benchmark of 296 bilingual questions. The benchmark targets four structural skills central to real research: following multi-hop evidence chains, verifying claims across sources, re-ordering fragmented information, and rejecting unsupported assumptions. We evaluate 16 frontier systems with human verification and find that these structural tasks remain highly challenging: the best system reaches only 19.9% average accuracy. The results show that strong agents can sometimes reorganize evidence and detect false premises, but still struggle with long-horizon synthesis and intersection-heavy verification. Beyond evaluation, the same theory also leads to practical system improvements. We instantiate theory-guided interventions such as tracked search, which preserves retrieval traces, and category tools, which add explicit verification and synthesis steps. These interventions yield measurable gains in API-based deep research systems. Our work therefore provides both a challenging benchmark and concrete design guidance for building more reliable research agents.

Keywords

Cite

@article{arxiv.2603.25342,
  title  = {From Intent to Evidence: A Categorical Approach for Structural Evaluation of Deep Research Agents},
  author = {Shuoling Liu and Zhiquan Tan and Kun Yi and Hui Wu and Yihan Li and Jiangpeng Yan and Liyuan Chen and Kai Chen and Qiang Yang},
  journal= {arXiv preprint arXiv:2603.25342},
  year   = {2026}
}