Technical report: Training Mixture Density Networks with full covariance matrices
Abstract
Mixture Density Networks are a tried and tested tool for modelling conditional probability distributions. As such, they constitute a great baseline for novel approaches to this problem. In the standard formulation, an MDN takes some input and outputs parameters for a Gaussian mixture model with restrictions on the mixture components' covariance. Since covariance between random variables is a central issue in the conditional modeling problems we were investigating, I derived and implemented an MDN formulation with unrestricted covariances. It is likely that this has been done before, but I could not find any resources online. For this reason, I have documented my approach in the form of this technical report, in hopes that it may be useful to others facing a similar situation.
Cite
@article{arxiv.2003.05739,
title = {Technical report: Training Mixture Density Networks with full covariance matrices},
author = {Jakob Kruse},
journal= {arXiv preprint arXiv:2003.05739},
year = {2020}
}