On Convergence and Generalization of Dropout Training

Poorya Mianjy; Raman Arora

On Convergence and Generalization of Dropout Training

Machine Learning 2020-10-27 v1 Machine Learning

Authors: Poorya Mianjy , Raman Arora

Abstract

We study dropout in two-layer neural networks with rectified linear unit (ReLU) activations. Under mild overparametrization and assuming that the limiting kernel can separate the data distribution with a positive margin, we show that dropout training with logistic loss achieves $\epsilon$ -suboptimality in test error in $O(1/\epsilon)$ iterations.

Keywords

activation functions dropout regularization neural networks

Cite

@article{arxiv.2010.12711,
  title  = {On Convergence and Generalization of Dropout Training},
  author = {Poorya Mianjy and Raman Arora},
  journal= {arXiv preprint arXiv:2010.12711},
  year   = {2020}
}

Related papers

View all related →