English
Related papers

Related papers: 2Direction: Theoretically Faster Distributed Train…

200 papers

A rich body of prior work has highlighted the existence of communication bottlenecks in synchronous data-parallel training. To alleviate these bottlenecks, a long line of recent work proposes gradient and model compression methods. In this…

Distributed, Parallel, and Cluster Computing · Computer Science 2021-07-01 Saurabh Agarwal , Hongyi Wang , Shivaram Venkataraman , Dimitris Papailiopoulos

We develop two new algorithms, called, FedDR and asyncFedDR, for solving a fundamental nonconvex composite optimization problem in federated learning. Our algorithms rely on a novel combination between a nonconvex Douglas-Rachford splitting…

Machine Learning · Statistics 2021-10-29 Quoc Tran-Dinh , Nhan H. Pham , Dzung T. Phan , Lam M. Nguyen

Federated learning can enable remote workers to collaboratively train a shared machine learning model while allowing training data to be kept locally. In the use case of wireless mobile devices, the communication overhead is a critical…

Distributed, Parallel, and Cluster Computing · Computer Science 2022-01-11 Kai Yue , Richeng Jin , Chau-Wai Wong , Huaiyu Dai

Distributed deep learning (DL) has become prevalent in recent years to reduce training time by leveraging multiple computing devices (e.g., GPUs/TPUs) due to larger models and datasets. However, system scalability is limited by…

Distributed, Parallel, and Cluster Computing · Computer Science 2023-09-04 Zhenheng Tang , Shaohuai Shi , Wei Wang , Bo Li , Xiaowen Chu

This paper presents a novel accelerated distributed algorithm for unconstrained consensus optimization over static undirected networks. The proposed algorithm combines the benefits of acceleration from momentum, the robustness of the…

Optimization and Control · Mathematics 2024-05-15 Eduardo Sebastián , Mauro Franceschelli , Andrea Gasparri , Eduardo Montijano , Carlos Sagüés

In this paper, we focus on an asynchronous distributed optimization problem. In our problem, each node is endowed with a convex local cost function, and is able to communicate with its neighbors over a directed communication network.…

Optimization and Control · Mathematics 2023-09-12 Apostolos I. Rikos , Wei Jiang , Themistoklis Charalambous , Karl H. Johansson

Decentralized distributed learning is the key to enabling large-scale machine learning (training) on edge devices utilizing private user-generated local data, without relying on the cloud. However, the practical realization of such…

Machine Learning · Computer Science 2022-09-13 Sai Aparna Aketi , Sangamesh Kodge , Kaushik Roy

In the large collection of existing distributed algorithms for convex multi-agent optimization, only a handful of them provide convergence rate guarantees on agent networks with time-varying topologies, which, however, restrict the problem…

Optimization and Control · Mathematics 2018-04-06 Xuyang Wu , Jie Lu

Distributed optimization is essential for training large models on large datasets. Multiple approaches have been proposed to reduce the communication overhead in distributed training, such as synchronizing only after performing multiple…

Machine Learning · Computer Science 2020-02-21 Jianyu Wang , Vinayak Tantia , Nicolas Ballas , Michael Rabbat

We study distributed optimization algorithms for minimizing the average of convex functions. The applications include empirical risk minimization problems in statistical machine learning where the datasets are large and have to be stored on…

Optimization and Control · Mathematics 2016-01-07 Jason D. Lee , Qihang Lin , Tengyu Ma , Tianbao Yang

Optimization problems on the Stiefel manifold, ranging from principal component analysis to enhancing neural network robustness, are ubiquitous in machine learning. The Landing algorithm avoids computationally expensive retraction…

Optimization and Control · Mathematics 2025-08-12 Yilong Song , Peijin Li , Bin Gao , Kun Yuan

Communication bottlenecks and the presence of stragglers pose significant challenges in distributed learning (DL). To deal with these challenges, recent advances leverage unbiased compression functions and gradient coding. However, the…

Distributed, Parallel, and Cluster Computing · Computer Science 2026-03-18 Chengxi Li , Ming Xiao , Mikael Skoglund

The exponential growth in model sizes has significantly increased the communication burden in Federated Learning (FL). Existing methods to alleviate this burden by transmitting compressed gradients often face high compression errors, which…

Machine Learning · Computer Science 2025-02-06 Yuhao Zhou , Yuxin Tian , Mingjia Shi , Yuanxi Li , Yanan Sun , Qing Ye , Jiancheng Lv

In this paper, we study the distributed nonconvex optimization problem, which aims to minimize the average value of the local nonconvex cost functions using local information exchange. To reduce the communication overhead, we introduce…

Optimization and Control · Mathematics 2025-02-12 Lei Xu , Xinlei Yi , Guanghui Wen , Yang Shi , Karl H. Johansson , Tao Yang

This paper considers the decentralized convex optimization problem, which has a wide range of applications in large-scale machine learning, sensor networks, and control theory. We propose novel algorithms that achieve optimal computation…

Machine Learning · Computer Science 2023-10-11 Haishan Ye , Luo Luo , Ziang Zhou , Tong Zhang

Federated Learning (FL) is a machine learning approach that enables the creation of shared models for powerful applications while allowing data to remain on devices. This approach provides benefits such as improved data privacy, security,…

Distributed, Parallel, and Cluster Computing · Computer Science 2023-04-25 Jieming Bian , Cong Shen , Jie Xu

Recent advances in machine learning, wireless communication, and mobile hardware technologies promisingly enable federated learning (FL) over massive mobile edge devices, which opens new horizons for numerous intelligent mobile…

Machine Learning · Computer Science 2020-12-23 Liang Li , Dian Shi , Ronghui Hou , Hui Li , Miao Pan , Zhu Han

As a prevalent distributed learning paradigm, Federated Learning (FL) trains a global model on a massive amount of devices with infrequent communication. This paper investigates a class of composite optimization and statistical recovery…

Machine Learning · Computer Science 2022-10-04 Yajie Bao , Michael Crawshaw , Shan Luo , Mingrui Liu

Federated Learning (FL) has recently received a lot of attention for large-scale privacy-preserving machine learning. However, high communication overheads due to frequent gradient transmissions decelerate FL. To mitigate the communication…

Machine Learning · Computer Science 2021-05-27 Milad Khademi Nori , Sangseok Yun , Il-Min Kim

We study the so-called distributed two-time-scale gradient method for solving convex optimization problems over a network of agents when the communication bandwidth between the nodes is limited, and so information that is exchanged between…

Systems and Control · Electrical Eng. & Systems 2021-06-01 Marcos M. Vasconcelos , Thinh T. Doan , Urbashi Mitra