English

Learning multivariate Gaussians with imperfect advice

Machine Learning 2025-02-03 v3 Data Structures and Algorithms Information Theory math.IT Machine Learning

Abstract

We revisit the problem of distribution learning within the framework of learning-augmented algorithms. In this setting, we explore the scenario where a probability distribution is provided as potentially inaccurate advice on the true, unknown distribution. Our objective is to develop learning algorithms whose sample complexity decreases as the quality of the advice improves, thereby surpassing standard learning lower bounds when the advice is sufficiently accurate. Specifically, we demonstrate that this outcome is achievable for the problem of learning a multivariate Gaussian distribution N(μ,Σ)N(\boldsymbol{\mu}, \boldsymbol{\Sigma}) in the PAC learning setting. Classically, in the advice-free setting, Θ~(d2/ε2)\tilde{\Theta}(d^2/\varepsilon^2) samples are sufficient and worst case necessary to learn dd-dimensional Gaussians up to TV distance ε\varepsilon with constant probability. When we are additionally given a parameter Σ~\tilde{\boldsymbol{\Sigma}} as advice, we show that O~(d2β/ε2)\tilde{O}(d^{2-\beta}/\varepsilon^2) samples suffices whenever Σ~1/2ΣΣ~1/2Id1εd1β\| \tilde{\boldsymbol{\Sigma}}^{-1/2} \boldsymbol{\Sigma} \tilde{\boldsymbol{\Sigma}}^{-1/2} - \boldsymbol{I_d} \|_1 \leq \varepsilon d^{1-\beta} (where 1\|\cdot\|_1 denotes the entrywise 1\ell_1 norm) for any β>0\beta > 0, yielding a polynomial improvement over the advice-free setting.

Keywords

Cite

@article{arxiv.2411.12700,
  title  = {Learning multivariate Gaussians with imperfect advice},
  author = {Arnab Bhattacharyya and Davin Choo and Philips George John and Themis Gouleakis},
  journal= {arXiv preprint arXiv:2411.12700},
  year   = {2025}
}
R2 v1 2026-06-28T20:05:19.890Z