Any Deep ReLU Network is Shallow
Machine Learning
2023-06-22 v1 Artificial Intelligence
Machine Learning
Abstract
We constructively prove that every deep ReLU network can be rewritten as a functionally identical three-layer network with weights valued in the extended reals. Based on this proof, we provide an algorithm that, given a deep ReLU network, finds the explicit weights of the corresponding shallow network. The resulting shallow network is transparent and used to generate explanations of the model s behaviour.
Keywords
Cite
@article{arxiv.2306.11827,
title = {Any Deep ReLU Network is Shallow},
author = {Mattia Jacopo Villani and Nandi Schoots},
journal= {arXiv preprint arXiv:2306.11827},
year = {2023}
}
Comments
12 pages including bibliography and appendix