A gradient descent perspective on Sinkhorn
Optimization and Control
2020-06-11 v3
Authors:
Flavien Léger
Abstract
We present a new perspective on the popular Sinkhorn algorithm, showing that it can be seen as a Bregman gradient descent (mirror descent) of a relative entropy (Kullback-Leibler divergence). This viewpoint implies a new sublinear convergence rate with a robust constant.
Cite
@article{arxiv.2002.03758,
title = {A gradient descent perspective on Sinkhorn},
author = {Flavien Léger},
journal= {arXiv preprint arXiv:2002.03758},
year = {2020}
}
Related papers
View all related →
Machine Learning · Computer Science
Sinkhorn Algorithm as a Special Case of Stochastic Mirror Descent
Konstantin Mishchenko
2019-09-17
Optimization and Control · Mathematics
Mirror Descent with Relative Smoothness in Measure Spaces, with application to Sinkhorn and EM
Pierre-Cyril Aubin-Frankowski, Anna Korba, Flavien Léger
2022-10-12
Machine Learning · Computer Science
Bregman-Hausdorff divergence: strengthening the connections between computational geometry and machine learning
Tuyen Pham, Hana Dal Poz Kouřimská, Hubert Wagner
2025-04-11
Machine Learning · Computer Science
Sparse Q-learning with Mirror Descent
Sridhar Mahadevan, Bo Liu
2012-10-19
Machine Learning · Computer Science
Sinkhorn Flow: A Continuous-Time Framework for Understanding and Generalizing the Sinkhorn Algorithm
Mohammad Reza Karimi, Ya-Ping Hsieh, Andreas Krause
2023-11-29
Machine Learning · Computer Science
Sinkhorn Barycenter via Functional Gradient Descent
Zebang Shen, Zhenfu Wang, Alejandro Ribeiro, Hamed Hassani
2020-07-22
Computation · Statistics
A connection between Tempering and Entropic Mirror Descent
Nicolas Chopin, Francesca R. Crucinio, Anna Korba
2024-06-18
Optimization and Control · Mathematics
Provable Phase Retrieval with Mirror Descent
Jean-Jacques Godeme, Jalal Fadili, Xavier Buet, Myriam Zerrad +2
2024-06-21
Machine Learning · Computer Science
Learning Sinkhorn divergences for supervised change point detection
Nauman Ahad, Eva L. Dyer, Keith B. Hengen, Yao Xie +1
2022-02-11
Machine Learning · Statistics
Implicit Regularization in Matrix Sensing via Mirror Descent
Fan Wu, Patrick Rebeschini
2021-10-28
Statistics Theory · Mathematics
Monotonically Decreasing Sequence of Divergences
Tomohiro Nishiyama
2019-10-22
Optimization and Control · Mathematics
Gradient descent in a generalised Bregman distance framework
Martin Benning, Marta M. Betcke, Matthias J. Ehrhardt, Carola-Bibiane Schönlieb
2016-12-28
Machine Learning · Statistics
On the geometry of Stein variational gradient descent
A. Duncan, N. Nuesken, L. Szpruch
2023-02-14
Optimization and Control · Mathematics
Gradient descent with a general cost
Flavien Léger, Pierre-Cyril Aubin-Frankowski
2023-06-13
Optimization and Control · Mathematics
Sinkhorn Distributionally Robust Optimization
Jie Wang, Rui Gao, Yao Xie
2025-03-27
Cryptography and Security · Computer Science
Unifying computational entropies via Kullback-Leibler divergence
Rohit Agrawal, Yi-Hsiu Chen, Thibaut Horel, Salil Vadhan
2024-11-15
Machine Learning · Computer Science
Gradients should stay on Path: Better Estimators of the Reverse- and Forward KL Divergence for Normalizing Flows
Lorenz Vaitl, Kim A. Nicoli, Shinichi Nakajima, Pan Kessel
2022-07-19
Optimization and Control · Mathematics
A Simple Convergence Analysis of Bregman Proximal Gradient Algorithm
Yi Zhou, Yingbin Liang, Lixin Shen
2017-12-19
Data Structures and Algorithms · Computer Science
A Relational Gradient Descent Algorithm For Support Vector Machine Training
Mahmoud Abo-Khamis, Sungjin Im, Benjamin Moseley, Kirk Pruhs +1
2020-05-13
Optimization and Control · Mathematics
Mirror Descent for Stochastic Control Problems with Measure-valued Controls
Bekzhan Kerimkulov, David Šiška, Łukasz Szpruch, Yufei Zhang
2025-08-22
Optimization and Control · Mathematics
Convergence Rate Bounds for the Mirror Descent Method: IQCs and the Bregman Divergence
Mengmou Li, Khaled Laib, Ioannis Lestas
2022-09-12
Optimization and Control · Mathematics
Convergence Rate Bounds for the Mirror Descent Method: IQCs, Popov Criterion and Bregman Divergence
Mengmou Li, Khaled Laib, Takeshi Hatanaka, Ioannis Lestas
2024-09-16
Information Theory · Computer Science
A Bregman Proximal Perspective on Classical and Quantum Blahut-Arimoto Algorithms
Kerry He, James Saunderson, Hamza Fawzi
2024-06-10
Optimization and Control · Mathematics
On the Convergence Rate of Projected Gradient Descent for a Back-Projection based Objective
Tom Tirer, Raja Giryes
2021-08-10
Probability · Mathematics
Wasserstein Mirror Gradient Flow as the limit of the Sinkhorn Algorithm
Nabarun Deb, Young-Heon Kim, Soumik Pal, Geoffrey Schiebinger
2026-04-21