Blind Descent: A Prequel to Gradient Descent
Abstract
We describe an alternative learning method for neural networks, which we call Blind Descent. By design, Blind Descent does not face problems like exploding or vanishing gradients. In Blind Descent, gradients are not used to guide the learning process. In this paper, we present Blind Descent as a more fundamental learning process compared to gradient descent. We also show that gradient descent can be seen as a specific case of the Blind Descent algorithm. We also train two neural network architectures, a multilayer perceptron and a convolutional neural network, using the most general Blind Descent algorithm to demonstrate a proof of concept.
Cite
@article{arxiv.2006.11505,
title = {Blind Descent: A Prequel to Gradient Descent},
author = {Akshat Gupta and Prasad N R},
journal= {arXiv preprint arXiv:2006.11505},
year = {2020}
}
Comments
We have discovered a new learning method which does random optimisation and solves the problem of initialisation for backpropagation in global optimisation. However, we also pose open questions that we discovered while performing experiments