Neural Tangent Kernels and Fisher Information Matrices for Simple ReLU Networks with Random Hidden Weights
Machine Learning
2025-07-28 v2 Machine Learning
Abstract
Fisher information matrices and neural tangent kernels (NTK) for 2-layer ReLU networks with random hidden weight are argued. We discuss the relation between both notions as a linear transformation and show that spectral decomposition of NTK with concrete forms of eigenfunctions with major eigenvalues. We also obtain an approximation formula of the functions presented by the 2-layer neural networks.
Keywords
Cite
@article{arxiv.2507.18555,
title = {Neural Tangent Kernels and Fisher Information Matrices for Simple ReLU Networks with Random Hidden Weights},
author = {Jun'ichi Takeuchi and Yoshinari Takeishi and Noboru Murata and Kazushi Mimura and Ka Long Keith Ho and Hiroshi Nagaoka},
journal= {arXiv preprint arXiv:2507.18555},
year = {2025}
}