English
Related papers

Related papers: Learned Gradient Compression for Distributed Deep …

200 papers

Neural networks are trained primarily based on their inputs and outputs, without regard for their internal mechanisms. These neglected mechanisms determine properties that are critical for safety, like (i) transparency; (ii) the absence of…

Machine Learning · Computer Science 2024-12-02 Alex Cloud , Jacob Goldman-Wetzler , Evžen Wybitul , Joseph Miller , Alexander Matt Turner

Communication overhead is a critical challenge in federated learning, particularly in bandwidth-constrained networks. Although many methods have been proposed to reduce communication overhead, most focus solely on compressing individual…

Machine Learning · Computer Science 2026-01-16 Shenlong Zheng , Zhen Zhang , Yuhui Deng , Geyong Min , Lin Cui

In this paper we present an end-to-end meta-learned system for image compression. Traditional machine learning based approaches to image compression train one or more neural network for generalization performance. However, at inference…

Image and Video Processing · Electrical Eng. & Systems 2021-05-04 Nannan Zou , Honglei Zhang , Francesco Cricri , Hamed R. Tavakoli , Jani Lainema , Miska Hannuksela , Emre Aksu , Esa Rahtu

Lossy gradient compression, with either unbiased or biased compressors, has become a key tool to avoid the communication bottleneck in centrally coordinated distributed training of machine learning models. We analyze the performance of two…

Machine Learning · Computer Science 2020-12-23 Sebastian U. Stich

Optimizing distributed learning systems is an art of balancing between computation and communication. There have been two lines of research that try to deal with slower networks: {\em communication compression} for low bandwidth networks,…

Machine Learning · Computer Science 2019-02-04 Hanlin Tang , Shaoduo Gan , Ce Zhang , Tong Zhang , Ji Liu

As an increasing amount of image and video content will be analyzed by machines, there is demand for a new codec paradigm that is capable of compressing visual input primarily for the purpose of computer vision inference, while secondarily…

Image and Video Processing · Electrical Eng. & Systems 2023-01-12 Ezgi Ozyilkan , Mateen Ulhaq , Hyomin Choi , Fabien Racape

With the rapid increase of big data, distributed Machine Learning (ML) has been widely applied in training large-scale models. Stochastic Gradient Descent (SGD) is arguably the workhorse algorithm of ML. Distributed ML models trained by SGD…

Machine Learning · Computer Science 2021-12-09 Keyu Yang , Lu Chen , Zhihao Zeng , Yunjun Gao

With the rapid growth of data, distributed momentum stochastic gradient descent~(DMSGD) has been widely used in distributed learning, especially for training large-scale deep models. Due to the latency and limited bandwidth of the network,…

Machine Learning · Statistics 2024-04-04 Chang-Wei Shi , Shen-Yi Zhao , Yin-Peng Xie , Hao Gao , Wu-Jun Li

The success of deep architectures is at least in part attributed to the layer-by-layer unsupervised pre-training that initializes the network. Various papers have reported extensive empirical analysis focusing on the design and…

Machine Learning · Computer Science 2015-02-13 Vamsi K Ithapu , Sathya Ravi , Vikas Singh

Deep feedforward and recurrent networks have achieved impressive results in many perception and language processing applications. This success is partially attributed to architectural innovations such as convolutional and long short-term…

Machine Learning · Statistics 2015-11-24 Arvind Neelakantan , Luke Vilnis , Quoc V. Le , Ilya Sutskever , Lukasz Kaiser , Karol Kurach , James Martens

Federated learning is a distributed learning method to train a shared model by aggregating the locally-computed gradient updates. In federated learning, bandwidth and privacy are two main concerns of gradient updates transmission. This…

Machine Learning · Computer Science 2019-08-23 Hongyu Li , Tianqi Han

Lateral inhibitory connections have been observed in the cortex of the biological brain, and has been extensively studied in terms of its role in cognitive functions. However, in the vanilla version of backpropagation in deep learning, all…

Computer Vision and Pattern Recognition · Computer Science 2022-08-16 Lei Jiang , Yongqing Liu , Shihai Xiao , Yansong Chua

Deep learning involves a difficult non-convex optimization problem with a large number of weights between any two adjacent layers of a deep structure. To handle large data sets or complicated networks, distributed training is needed, but…

Communication overhead is a major bottleneck hampering the scalability of distributed machine learning systems. Recently, there has been a surge of interest in using gradient compression to improve the communication efficiency of…

Machine Learning · Computer Science 2019-10-29 Shuai Zheng , Ziyue Huang , James T. Kwok

Over the last decade, deep learning has shown great success at performing computer vision tasks, including classification, super-resolution, and style transfer. Now, we apply it to data compression to help build the next generation of…

Image and Video Processing · Electrical Eng. & Systems 2024-09-16 Mateen Ulhaq

Deep learning is typically performed by learning a neural network solely from data in the form of input-output pairs ignoring available domain knowledge. In this work, the Constraint Guided Gradient Descent (CGGD) framework is proposed that…

Artificial Intelligence · Computer Science 2022-06-15 Quinten Van Baelen , Peter Karsmakers

There has been much interest in deploying deep learning algorithms on low-powered devices, including smartphones, drones, and medical sensors. However, full-scale deep neural networks are often too resource-intensive in terms of energy and…

Computer Vision and Pattern Recognition · Computer Science 2022-09-20 Yoshitomo Matsubara , Ruihan Yang , Marco Levorato , Stephan Mandt

Traditional CNN models are trained and tested on relatively low resolution images (<300 px), and cannot be directly operated on large-scale images due to compute and memory constraints. We propose Patch Gradient Descent (PatchGD), an…

Computer Vision and Pattern Recognition · Computer Science 2023-02-01 Deepak K. Gupta , Gowreesh Mago , Arnav Chavan , Dilip K. Prasad

Stochastic gradient descent (SGD) has achieved great success in training deep neural network, where the gradient is computed through back-propagation. However, the back-propagated values of different layers vary dramatically. This…

Machine Learning · Statistics 2018-02-28 Huishuai Zhang , Wei Chen , Tie-Yan Liu

Federated learning (FL) enables the training of a model leveraging decentralized data in client sites while preserving privacy by not collecting data. However, one of the significant challenges of FL is limited computation and low…

Machine Learning · Computer Science 2023-04-18 Riyasat Ohib , Bishal Thapaliya , Pratyush Gaggenapalli , Jingyu Liu , Vince Calhoun , Sergey Plis