Communication is one of the bottlenecks of distributed optimisation and learning. To overcome this bottleneck, we propose a novel quantization method that transforms a vector into a sample of components' indices drawn from a categorical distribution with probabilities proportional to values at those components. Then, we propose a primal and a primal-dual accelerated stochastic gradient methods that use our proposed quantization, and derive their convergence rates in terms of probabilities of large deviations. We focus on affine-constrained convex optimisation and its application to decentralised distributed optimisation problems. To illustrate the work of our algorithm, we apply it to the decentralised computation of semi-discrete entropy regularized Wasserstein barycenters.
@article{arxiv.2501.18312,
title = {Decentralised convex optimisation with probability-proportional-to-size quantization},
author = {Dmitrii Pasechniuk and Pavel Dvurechensky and César A. Uribe and Alexander Gasnikov},
journal= {arXiv preprint arXiv:2501.18312},
year = {2025}
}