Graph-SND: Sparse Aggregation for Behavioral Diversity in Multi-Agent Reinforcement Learning
Abstract
System Neural Diversity (SND) measures behavioral heterogeneity in multi-agent reinforcement learning by averaging pairwise distances over all agent pairs, making each call quadratic in team size. We introduce Graph-SND, which replaces this complete-graph average with a weighted average over the edges of an arbitrary graph . Three regimes follow: recovers SND exactly; a fixed sparse defines a localized diversity measure at cost; and random edge samples yield an unbiased Horvitz-Thompson estimator and a normalized sample mean with concentration in the sampled edge count . For fixed sparse graphs we prove forwarding-index distortion bounds for expanders and a spectral refinement under low-rank distance structure; for random -regular graphs we prove an unconditional probabilistic bound. On VMAS we verify recovery, unbiasedness, concentration, and wall-clock scaling, with a PettingZoo TVD panel checking non-Gaussian transfer. In a 500-iteration PPO run, Bernoulli- Graph-SND tracks full SND while reducing per-call metric time by about , and frozen-policy GPU timing up to follows the predicted speedup. Random -regular expanders empirically achieve at edges. In DiCo diversity control at , Bernoulli- Graph-SND preserves set-point tracking with paired reward differences indistinguishable from zero across nine matched cells while cutting per-call metric cost by . Together, these results show that the SND aggregation bottleneck can be removed without changing the metric's semantics, yielding a drop-in sparse alternative that scales beyond complete-graph SND and supports both passive measurement and closed-loop diversity control.
Cite
@article{arxiv.2605.05020,
title = {Graph-SND: Sparse Aggregation for Behavioral Diversity in Multi-Agent Reinforcement Learning},
author = {Shawn Ray},
journal= {arXiv preprint arXiv:2605.05020},
year = {2026}
}
Comments
22 pages, 12 figures, 7 tables