Quadratic number of nodes is sufficient to learn a dataset via gradient descent
Optimization and Control
2019-11-14 v1 Machine Learning
Statistics Theory
Statistics Theory
Abstract
We prove that if an activation function satisfies some mild conditions and number of neurons in a two-layered fully connected neural network with this activation function is beyond a certain threshold, then gradient descent on quadratic loss function finds the optimal weights of input layer for global minima in linear time. This threshold value is an improvement over previously obtained values. We hypothesise that this bound cannot be improved by the method we are using in this work.
Keywords
Cite
@article{arxiv.1911.05402,
title = {Quadratic number of nodes is sufficient to learn a dataset via gradient descent},
author = {Biswarup Das and Eugene. A. Golikov},
journal= {arXiv preprint arXiv:1911.05402},
year = {2019}
}
Comments
Machine learning using neural networks, gradient descent, optimization, overparametrization regime