On Training Deep Boltzmann Machines
Neural and Evolutionary Computing
2012-03-21 v1 Artificial Intelligence
Machine Learning
Abstract
The deep Boltzmann machine (DBM) has been an important development in the quest for powerful "deep" probabilistic models. To date, simultaneous or joint training of all layers of the DBM has been largely unsuccessful with existing training methods. We introduce a simple regularization scheme that encourages the weight vectors associated with each hidden unit to have similar norms. We demonstrate that this regularization can be easily combined with standard stochastic maximum likelihood to yield an effective training strategy for the simultaneous training of all layers of the deep Boltzmann machine.
Cite
@article{arxiv.1203.4416,
title = {On Training Deep Boltzmann Machines},
author = {Guillaume Desjardins and Aaron Courville and Yoshua Bengio},
journal= {arXiv preprint arXiv:1203.4416},
year = {2012}
}