Vulnerable Agent Identification in Large-Scale Multi-Agent Reinforcement Learning
Abstract
Partial agent failure becomes inevitable when systems scale up, making it crucial to identify the subset of agents whose failure causes worst-case system performance degradations. We study this Vulnerable Agent Identification (VAI) problem in large-scale multi-agent reinforcement learning (MARL). We frame VAI as a Hierarchical Adversarial Decentralized Mean Field Control (HAD-MFC), where the upper level selects vulnerable agents as an NP-hard task and the lower level learns their worst-case adversarial policies via mean-field MARL. The two problems are coupled together, making HAD-MFC difficult to solve. To handle this, we first decouple the hierarchical process by Fenchel-Rockafellar transform, resulting a regularized mean-field Bellman operator for upper level that enables independent learning at each level, thus reducing computational complexity. We next reformulate the upper-level NP-hard problem as an MDP with dense rewards, allowing sequential identification of vulnerable agents via greedy and RL algorithms. This decomposition provably preserves the optimal solution. Experiments show our method effectively identifies more vulnerable agents in large-scale MARL and the rule-based system, fooling system into worse failures, and reveals the vulnerability of each agent in large systems. Code available at https://github.com/Waken-dream/VAI
Cite
@article{arxiv.2509.15103,
title = {Vulnerable Agent Identification in Large-Scale Multi-Agent Reinforcement Learning},
author = {Simin Li and Zihao Mao and Zheng Yuwei and Linhao Wang and Ruixiao Xu and Chengdong Ma and Zhiqian Liu and Xin Yu and Yuqing Ma and Xin Wang and Jie Luo and Bo An and Yaodong Yang and Weifeng Lv and Xianglong Liu},
journal= {arXiv preprint arXiv:2509.15103},
year = {2026}
}
Comments
Accepted by ICML 2026