Sample Complexity of Probability Divergences under Group Symmetry
Abstract
We rigorously quantify the improvement in the sample complexity of variational divergence estimations for group-invariant distributions. In the cases of the Wasserstein-1 metric and the Lipschitz-regularized -divergences, the reduction of sample complexity is proportional to the group size if the group is finite. In addition to the published version at ICML 2023, our proof indeed has included the case when the group is infinite such as compact Lie groups, the convergence rate can be further improved and depends on the intrinsic dimension of the fundamental domain characterized by the scaling of its covering number. Our approach is different from that in [Tahmasebi & Jegelka, ICML 2024] and our work also applies to asymmetric divergences, such as the Lipschitz-regularized -divergences. For the maximum mean discrepancy (MMD), the improvement of sample complexity is more nuanced, as it depends on not only the group size but also the choice of kernel. Numerical simulations verify our theories.
Cite
@article{arxiv.2302.01915,
title = {Sample Complexity of Probability Divergences under Group Symmetry},
author = {Ziyu Chen and Markos A. Katsoulakis and Luc Rey-Bellet and Wei Zhu},
journal= {arXiv preprint arXiv:2302.01915},
year = {2024}
}
Comments
In addition to our published version at ICML 2023, we include the case when the group is infinite such as compact Lie groups. Our approach is different from that in [Tahmasebi & Jegelka, ICML 2024] and our work also applies to asymmetric divergences, such as the Lipschitz-regularized $\alpha$-divergences