Effects of the optimisation of the margin distribution on generalisation in deep architectures
Machine Learning
2017-04-20 v1
Abstract
Despite being so vital to success of Support Vector Machines, the principle of separating margin maximisation is not used in deep learning. We show that minimisation of margin variance and not maximisation of the margin is more suitable for improving generalisation in deep architectures. We propose the Halfway loss function that minimises the Normalised Margin Variance (NMV) at the output of a deep learning models and evaluate its performance against the Softmax Cross-Entropy loss on the MNIST, smallNORB and CIFAR-10 datasets.
Keywords
Cite
@article{arxiv.1704.05646,
title = {Effects of the optimisation of the margin distribution on generalisation in deep architectures},
author = {Lech Szymanski and Brendan McCane and Wei Gao and Zhi-Hua Zhou},
journal= {arXiv preprint arXiv:1704.05646},
year = {2017}
}