English
Related papers

Related papers: A Distributed SGD Algorithm with Global Sketching …

200 papers

Machine learning models trained with \emph{stochastic} gradient descent (SGD) can generalize better than those trained with deterministic gradient descent (GD). In this work, we study SGD's impact on generalization through the lens of the…

Machine Learning · Computer Science 2025-12-09 Hongjian Lan , Yucong Liu , Florian Schäfer

We first propose a decentralized proximal stochastic gradient tracking method (DProxSGT) for nonconvex stochastic composite problems, with data heterogeneously distributed on multiple workers in a decentralized connected network. To save…

Optimization and Control · Mathematics 2023-03-01 Yonggui Yan , Jie Chen , Pin-Yu Chen , Xiaodong Cui , Songtao Lu , Yangyang Xu

In this work, we study distributed sketching methods for large scale regression problems. We leverage multiple randomized sketches for reducing the problem dimensions as well as preserving privacy and improving straggler resilience in…

Distributed, Parallel, and Cluster Computing · Computer Science 2020-06-23 Burak Bartan , Mert Pilanci

First-order methods like stochastic gradient descent(SGD) are recently the popular optimization method to train deep neural networks (DNNs), but second-order methods are scarcely used because of the overpriced computing cost in getting the…

Machine Learning · Computer Science 2021-04-01 Jingcheng Zhou , Wei Wei , Zhiming Zheng

Deep unfolding networks have recently gained popularity in the context of solving imaging inverse problems. However, the computational and memory complexity of data-consistency layers within traditional deep unfolding networks scales with…

Image and Video Processing · Electrical Eng. & Systems 2021-06-04 Jiaming Liu , Yu Sun , Weijie Gan , Xiaojian Xu , Brendt Wohlberg , Ulugbek S. Kamilov

Stochastic Gradient Descent (SGD) is a popular optimization method which has been applied to many important machine learning tasks such as Support Vector Machines and Deep Neural Networks. In order to parallelize SGD, minibatch training is…

Machine Learning · Statistics 2014-05-14 Peilin Zhao , Tong Zhang

Decentralized learning offers privacy and communication efficiency when data are naturally distributed among agents communicating over an underlying graph. Motivated by overparameterized learning settings, in which models are trained to…

Machine Learning · Computer Science 2023-03-28 Hossein Taheri , Christos Thrampoulidis

Many popular distributed optimization methods for training machine learning models fit the following template: a local gradient estimate is computed independently by each worker, then communicated to a master, which subsequently performs…

Machine Learning · Computer Science 2019-06-05 Konstantin Mishchenko , Filip Hanzely , Peter Richtárik

Communication compression is a crucial technique for modern distributed learning systems to alleviate their communication bottlenecks over slower networks. Despite recent intensive studies of gradient compression for data parallel-style…

Machine Learning · Computer Science 2023-03-08 Jue Wang , Binhang Yuan , Luka Rimanic , Yongjun He , Tri Dao , Beidi Chen , Christopher Re , Ce Zhang

This paper considers decentralized optimization with application to machine learning on graphs. The growing size of neural network (NN) models has motivated prior works on decentralized stochastic gradient algorithms to incorporate…

Optimization and Control · Mathematics 2021-10-12 Arjun Ashok Rao , Hoi-To Wai

Distributed Deep Learning (DDL) is essential for large-scale Deep Learning (DL) training. Synchronous Stochastic Gradient Descent (SSGD) 1 is the de facto DDL optimization method. Using a sufficiently large batch size is critical to…

Machine Learning · Computer Science 2021-12-03 Wei Zhang , Mingrui Liu , Yu Feng , Xiaodong Cui , Brian Kingsbury , Yuhai Tu

Edge machine learning involves the deployment of learning algorithms at the wireless network edge so as to leverage massive mobile data for enabling intelligent applications. The mainstream edge learning approach, federated learning, has…

Information Theory · Computer Science 2020-06-24 Yuqing Du , Sheng Yang , Kaibin Huang

An increasing bottleneck in decentralized optimization is communication. Bigger models and growing datasets mean that decentralization of computation is important and that the amount of information exchanged is quickly growing. While…

Machine Learning · Computer Science 2021-08-19 Tharindu B. Adikari , Stark C. Draper

Many applications today, such as NLP, network analysis, and code analysis, rely on semantically embedding objects into low-dimensional fixed-length vectors. Such embeddings naturally provide a way to perform useful downstream tasks, such as…

Machine Learning · Computer Science 2020-02-25 Gurbinder Gill , Roshan Dathathri , Saeed Maleki , Madan Musuvathi , Todd Mytkowicz , Olli Saarikivi

The implementation of a vast majority of machine learning (ML) algorithms boils down to solving a numerical optimization problem. In this context, Stochastic Gradient Descent (SGD) methods have long proven to provide good results, both in…

Distributed, Parallel, and Cluster Computing · Computer Science 2015-10-06 Janis Keuper , Franz-Josef Pfreundt

Deep learning networks are typically trained by Stochastic Gradient Descent (SGD) methods that iteratively improve the model parameters by estimating a gradient on a very small fraction of the training data. A major roadblock faced when…

Machine Learning · Computer Science 2020-06-11 Tao Lin , Lingjing Kong , Sebastian U. Stich , Martin Jaggi

In this paper, we consider solving the distributed optimization problem over a multi-agent network under the communication restricted setting. We study a compressed decentralized stochastic gradient method, termed ``compressed exact…

Optimization and Control · Mathematics 2024-10-01 Kun Huang , Shi Pu

Distributed methods are essential for handling machine learning pipelines comprising large-scale models and datasets. However, their benefits often come at the cost of increased communication overhead between the central server and agents,…

Machine Learning · Computer Science 2025-03-03 Enea Monzio Compagnoni , Rustem Islamov , Frank Norbert Proske , Aurelien Lucchi

We consider a decentralized learning problem, where a set of computing nodes aim at solving a non-convex optimization problem collaboratively. It is well-known that decentralized optimization schemes face two major system bottlenecks:…

Machine Learning · Computer Science 2019-11-04 Amirhossein Reisizadeh , Hossein Taheri , Aryan Mokhtari , Hamed Hassani , Ramtin Pedarsani

Many supervised learning tasks have intrinsic symmetries, such as translational and rotational symmetry in image classifications. These symmetries can be exploited to enhance performance. We formulate the symmetry constraints into a concise…

Quantum Physics · Physics 2024-08-14 Kaiming Bian , Shitao Zhang , Fei Meng , Wen Zhang , Oscar Dahlsten