规模不变的 regret matching 与零和博弈中的最优收敛在线学习：理论与实践的桥梁
摘要
对于通过一阶方法求解零和博弈的理论与实践之间，已有数十年的差距。尽管已确立 的收敛率，但实际中最有效的范畴是基于 regret matching 及其现代变体的反事实后悔最小化（CFR）。特别是，大多数基准测试的最佳实践是预测后悔匹配（PRM）。然而，这类算法即使在自我对弈中也可能表现出较慢的 收敛。本文在理论与实践之间架起桥梁。我们提出了一种新的规模不变且无参数的 PRM 变体，称为 IREG-PRM。我们证明它实现了 最佳迭代和 （即最优）平均迭代收敛保证，同时在大多数基准博弈中与 PRM 等价甚至更好。从技术角度看，我们将 (IREG-)PRM 与带自适应学习率的乐观梯度下降类比。反映这一理论桥梁，我们发现我们所考虑的自适应乐观梯度下降版本与 IREG-PRM 表现相当。This demystifies the effectiveness of the regret matching family vis-a-vis more standard optimization techniques. Moreover, we extend our analysis beyond zero-sum games to a family of variational inequality problems that includes harmonic games, as well as extensive-form games with fully-mixed equilibria, via a new and intriguing connection between CFR and harmonic games. Unlike prior work in harmonic games, our algorithms do not require knowing the underlying weights by virtue of scale invariance. Under the weighted Minty condition, we show that any algorithm satisfying a scale-invariant RVU property (such as IREG-PRM) has constant regret (in self-play) and iterate convergence.
引用
@article{arxiv.2510.04407,
title = {Scale-Invariant Regret Matching and Online Learning with Optimal Convergence: Bridging Theory and Practice in Zero-Sum Games},
author = {Brian Hu Zhang and Ioannis Anagnostides and Tuomas Sandholm},
journal= {arXiv preprint arXiv:2510.04407},
year = {2026}
}
备注
Compared to the previous version, this version includes new results on harmonic games and extensive-form games. Abstract abridged due to arXiv length constraints