Talk to Right Specialists: Iterative Routing in Multi-agent Systems for Question Answering
Abstract
Retrieval-augmented generation (RAG) agents are increasingly deployed to answer questions over local knowledge bases that cannot be centralized due to knowledge-sovereignty constraints. This results in two recurring failures in production: users do not know which agent to consult, and complex questions require evidence distributed across multiple agents. To overcome these challenges, we propose RIRS, a training-free orchestration framework to enable a multi-agent system for question answering. In detail, RIRS summarizes each agent's local corpus in an embedding space, enabling a user-facing server to route queries only to the most relevant agents, reducing latency and avoiding noisy "broadcast-to-all" contexts. For complicated questions, the server can iteratively aggregate responses to derive intermediate results and refine the question to bridge the gap toward a comprehensive answer. Extensive experiments demonstrate the effectiveness of RIRS, including its ability to precisely select agents and provide accurate responses to single-hop queries, and its use of an iterative strategy to achieve accurate, multi-step resolutions for complex queries.
Cite
@article{arxiv.2501.07813,
title = {Talk to Right Specialists: Iterative Routing in Multi-agent Systems for Question Answering},
author = {Feijie Wu and Zitao Li and Fei Wei and Yaliang Li and Bolin Ding and Jing Gao},
journal= {arXiv preprint arXiv:2501.07813},
year = {2026}
}
Comments
Differences between v1 & v2: The algorithm name of the first version is RopMura, which decomposes a multi-hop query into several simple subqueries, and a question selector selects one of the subqueries to answer. In the second version, the name is updated to RIRS, which directly routes a query to the appropriate agents, regardless of whether the query is single-hop or multi-hop