Intelligent Matrix Exponentiation
Machine Learning
2020-08-11 v1 Neural and Evolutionary Computing
Representation Theory
Machine Learning
Abstract
We present a novel machine learning architecture that uses the exponential of a single input-dependent matrix as its only nonlinearity. The mathematical simplicity of this architecture allows a detailed analysis of its behaviour, providing robustness guarantees via Lipschitz bounds. Despite its simplicity, a single matrix exponential layer already provides universal approximation properties and can learn fundamental functions of the input, such as periodic functions or multivariate polynomials. This architecture outperforms other general-purpose architectures on benchmark problems, including CIFAR-10, using substantially fewer parameters.
Cite
@article{arxiv.2008.03936,
title = {Intelligent Matrix Exponentiation},
author = {Thomas Fischbacher and Iulia M. Comsa and Krzysztof Potempa and Moritz Firsching and Luca Versari and Jyrki Alakuijala},
journal= {arXiv preprint arXiv:2008.03936},
year = {2020}
}
Comments
20 pages, 10 figures