English

Autoregressive Energy Machines

Machine Learning 2019-04-12 v1 Machine Learning

Abstract

Neural density estimators are flexible families of parametric models which have seen widespread use in unsupervised machine learning in recent years. Maximum-likelihood training typically dictates that these models be constrained to specify an explicit density. However, this limitation can be overcome by instead using a neural network to specify an energy function, or unnormalized density, which can subsequently be normalized to obtain a valid distribution. The challenge with this approach lies in accurately estimating the normalizing constant of the high-dimensional energy function. We propose the Autoregressive Energy Machine, an energy-based model which simultaneously learns an unnormalized density and computes an importance-sampling estimate of the normalizing constant for each conditional in an autoregressive decomposition. The Autoregressive Energy Machine achieves state-of-the-art performance on a suite of density-estimation tasks.

Keywords

Cite

@article{arxiv.1904.05626,
  title  = {Autoregressive Energy Machines},
  author = {Charlie Nash and Conor Durkan},
  journal= {arXiv preprint arXiv:1904.05626},
  year   = {2019}
}
R2 v1 2026-06-23T08:36:35.725Z