A Telescopic Binary Learning Machine for Training Neural Networks
Abstract
This paper proposes a new algorithm based on multi-scale stochastic local search with binary representation for training neural networks. In particular, we study the effects of neighborhood evaluation strategies, the effect of the number of bits per weight and that of the maximum weight range used for mapping binary strings to real values. Following this preliminary investigation, we propose a telescopic multi-scale version of local search where the number of bits is increased in an adaptive manner, leading to a faster search and to local minima of better quality. An analysis related to adapting the number of bits in a dynamic way is also presented. The control on the number of bits, which happens in a natural manner in the proposed method, is effective to increase the generalization performance. Benchmark tasks include a highly non-linear artificial problem, a control problem requiring either feed-forward or recurrent architectures for feedback control, and challenging real-world tasks in different application domains. The results demonstrate the effectiveness of the proposed method.
Cite
@article{arxiv.1509.00174,
title = {A Telescopic Binary Learning Machine for Training Neural Networks},
author = {Mauro Brunato and Roberto Battiti},
journal= {arXiv preprint arXiv:1509.00174},
year = {2016}
}
Comments
Submitted to IEEE Transactions on Neural Networks and Learning Systems, special issue on New Developments in Neural Network Structures for Signal Processing, Autonomous Decision, and Adaptive Control