AdaFamily: A family of Adam-like adaptive gradient methods

Hannes Fassold

AdaFamily: A family of Adam-like adaptive gradient methods

Machine Learning 2022-03-04 v1 Optimization and Control

Authors: Hannes Fassold

Abstract

We propose AdaFamily, a novel method for training deep neural networks. It is a family of adaptive gradient methods and can be interpreted as sort of a blend of the optimization algorithms Adam, AdaBelief and AdaMomentum. We perform experiments on standard datasets for image classification, demonstrating that our proposed method outperforms these algorithms.

Keywords

adaptive gradient methods adam adabelief adamomentum deep neural networks image classification

Cite

@article{arxiv.2203.01603,
  title  = {AdaFamily: A family of Adam-like adaptive gradient methods},
  author = {Hannes Fassold},
  journal= {arXiv preprint arXiv:2203.01603},
  year   = {2022}
}

Comments

submitted for ISPR 2022 conference

Related papers

View all related →