Multivariate Gaussian Variational Inference by Natural Gradient Descent
Machine Learning
2020-10-20 v2 Machine Learning
Robotics
Statistics Theory
Statistics Theory
Abstract
This short note reviews so-called Natural Gradient Descent (NGD) for multivariate Gaussians. The Fisher Information Matrix (FIM) is derived for several different parameterizations of Gaussians. Careful attention is paid to the symmetric nature of the covariance matrix when calculating derivatives. We show that there are some advantages to choosing a parameterization comprising the mean and inverse covariance matrix and provide a simple NGD update that accounts for the symmetric (and sparse) nature of the inverse covariance matrix.
Keywords
Cite
@article{arxiv.2001.10025,
title = {Multivariate Gaussian Variational Inference by Natural Gradient Descent},
author = {Timothy D. Barfoot},
journal= {arXiv preprint arXiv:2001.10025},
year = {2020}
}
Comments
11 pages, 0 figures; second version fixed a single typo