Functional Optimisation of Online Algorithms in Multilayer Neural Networks
Disordered Systems and Neural Networks
2009-10-30 v1
Abstract
We study the online dynamics of learning in fully connected soft committee machines in the student-teacher scenario. The locally optimal modulation function, which determines the learning algorithm, is obtained from a variational argument in such a manner as to maximise the average generalisation error decay per example. Simulations results for the resulting algorithm are presented for a few cases. The symmetric phase plateaux are found to be vastly reduced in comparison to those found when online backpropagation algorithms are used. A discussion of the implementation of these ideas as practical algorithms is given.
Cite
@article{arxiv.cond-mat/9706015,
title = {Functional Optimisation of Online Algorithms in Multilayer Neural Networks},
author = {Renato Vicente and Nestor Caticha},
journal= {arXiv preprint arXiv:cond-mat/9706015},
year = {2009}
}