Unifying mirror descent and dual averaging

Anatoli Juditsky; Joon Kwon; Éric Moulines

Unifying mirror descent and dual averaging

Optimization and Control 2022-06-13 v4 Machine Learning

Authors: Anatoli Juditsky , Joon Kwon , Éric Moulines

Abstract

We introduce and analyze a new family of first-order optimization algorithms which generalizes and unifies both mirror descent and dual averaging. Within the framework of this family, we define new algorithms for constrained optimization that combines the advantages of mirror descent and dual averaging. Our preliminary simulation study shows that these new algorithms significantly outperform available methods in some situations.

Keywords

optimization stochastic optimization primal-dual optimization

Cite

@article{arxiv.1910.13742,
  title  = {Unifying mirror descent and dual averaging},
  author = {Anatoli Juditsky and Joon Kwon and Éric Moulines},
  journal= {arXiv preprint arXiv:1910.13742},
  year   = {2022}
}

Related papers

View all related →

Machine Learning · Computer Science

A Generalized Online Mirror Descent with Applications to Classification and Regression

Francesco Orabona, Koby Crammer, Nicolò Cesa-Bianchi

2015-07-03

Machine Learning · Computer Science

Duality between subgradient and conditional gradient methods

Francis Bach

2013-10-21

Machine Learning · Computer Science

Faster Margin Maximization Rates for Generic and Adversarially Robust Optimization Methods

Guanghui Wang, Zihao Hu, Claudio Gentile, Vidya Muthukumar +1