On the validity of kernel approximations for orthogonally-initialized neural networks
Machine Learning
2021-04-14 v1 Neural and Evolutionary Computing
Abstract
In this note we extend kernel function approximation results for neural networks with Gaussian-distributed weights to single-layer networks initialized using Haar-distributed random orthogonal matrices (with possible rescaling). This is accomplished using recent results from random matrix theory.
Keywords
Cite
@article{arxiv.2104.05878,
title = {On the validity of kernel approximations for orthogonally-initialized neural networks},
author = {James Martens},
journal= {arXiv preprint arXiv:2104.05878},
year = {2021}
}