Generalization of the Change of Variables Formula with Applications to Residual Flows
Machine Learning
2021-07-12 v1 Machine Learning
Dynamical Systems
Abstract
Normalizing flows leverage the Change of Variables Formula (CVF) to define flexible density models. Yet, the requirement of smooth transformations (diffeomorphisms) in the CVF poses a significant challenge in the construction of these models. To enlarge the design space of flows, we introduce -diffeomorphisms as generalized transformations which may violate these requirements on zero Lebesgue-measure sets. This relaxation allows e.g. the use of non-smooth activation functions such as ReLU. Finally, we apply the obtained results to planar, radial, and contractive residual flows.
Keywords
Cite
@article{arxiv.2107.04346,
title = {Generalization of the Change of Variables Formula with Applications to Residual Flows},
author = {Niklas Koenen and Marvin N. Wright and Peter Maaß and Jens Behrmann},
journal= {arXiv preprint arXiv:2107.04346},
year = {2021}
}