English
Related papers

Related papers: Collective Communication Profiling of Modern-day M…

200 papers

Large Language Models increasingly rely on distributed techniques for their training and inference. These techniques require communication across devices which can reduce scaling efficiency as the number of devices increases. While some…

Hardware Architecture · Computer Science 2024-01-31 Suchita Pati , Shaizeen Aga , Mahzabeen Islam , Nuwan Jayasena , Matthew D. Sinclair

Nowadays the number of available processing cores within computing nodes which are used in recent clustered environments, are growing up with a rapid rate. Despite this trend, the number of available network interfaces in such computing…

Distributed, Parallel, and Cluster Computing · Computer Science 2012-07-13 Mohsen Soryani , Morteza Analoui , Ghobad Zarrinchian

The emerging applications of machine learning algorithms on mobile devices motivate us to offload the computation tasks of training a model or deploying a trained one to the cloud or at the edge of the network. One of the major challenges…

Machine Learning · Statistics 2020-07-28 Hamidreza Ehteram , Mohammad Ali Maddah-Ali , Mahtab Mirmohseni

There is a growing interest in development of in-network dispersed computing paradigms that leverage the computing capabilities of heterogeneous resources dispersed across the network for processing massive amount of data is collected at…

Distributed, Parallel, and Cluster Computing · Computer Science 2019-05-28 Chien-Sheng Yang , Ramtin Pedarsani , A. Salman Avestimehr

Large language models (LLMs) are increasingly used in the creation of online content, creating feedback loops as subsequent generations of models will be trained on this synthetic data. Such loops were shown to lead to distribution shifts -…

Machine Learning · Computer Science 2025-12-29 Grgur Kovač , Jérémy Perez , Rémy Portelas , Peter Ford Dominey , Pierre-Yves Oudeyer

We study scheduling control of parallel processing networks in which some resources need to simultaneously collaborate to perform some activities and some resources multitask. Resource collaboration and multitasking give rise to…

Optimization and Control · Mathematics 2020-12-29 Erhun Özkan

With an increasing number of smart devices like internet of things (IoT) devices deployed in the field, offloadingtraining of neural networks (NNs) to a central server becomes more and more infeasible. Recent efforts toimprove users'…

Machine Learning · Computer Science 2023-07-19 Kilian Pfeiffer , Martin Rapp , Ramin Khalili , Jörg Henkel

We introduce collaborative learning in which multiple classifier heads of the same network are simultaneously trained on the same training data to improve generalization and robustness to label noise with no extra inference cost. It…

Machine Learning · Statistics 2018-11-08 Guocong Song , Wei Chai

Federated learning (FL) is a highly pursued machine learning technique that can train a model centrally while keeping data distributed. Distributed computation makes FL attractive for bandwidth limited applications especially in wireless…

Machine Learning · Computer Science 2020-06-24 Xiang Ma , Haijian Sun , Rose Qingyang Hu

Predicting node labels on a given graph is a widely studied problem with many applications, including community detection and molecular graph prediction. This paper considers predicting multiple node labeling functions on graphs…

Machine Learning · Computer Science 2024-03-18 Dongyue Li , Haotian Ju , Aneesh Sharma , Hongyang R. Zhang

Language models suffer from various degenerate behaviors. These differ between tasks: machine translation (MT) exhibits length bias, while tasks like story generation exhibit excessive repetition. Recent work has attributed the difference…

Computation and Language · Computer Science 2022-10-21 Darcey Riley , David Chiang

We investigate multi-task learning approaches that use a shared feature representation for all tasks. To better understand the transfer of task information, we study an architecture with a shared module for all tasks and a separate output…

Machine Learning · Computer Science 2020-05-05 Sen Wu , Hongyang R. Zhang , Christopher Ré

We study the compressive diffusion strategies over distributed networks based on the diffusion implementation and adaptive extraction of the information from the compressed diffusion data. We demonstrate that one can achieve a comparable…

Systems and Control · Computer Science 2015-06-18 Muhammed O. Sayin , Suleyman S. Kozat

We present a novel federated multi-task learning method that leverages cross-client similarity to enable personalized learning for each client. To avoid transmitting the entire model to the parameter server, we propose a…

Machine Learning · Computer Science 2025-06-13 Ahmed Elbakary , Chaouki Ben Issaid , Mehdi Bennis

MapReduce is a commonly used framework for executing data-intensive jobs on distributed server clusters. We introduce a variant implementation of MapReduce, namely "Coded MapReduce", to substantially reduce the inter-server communication…

Distributed, Parallel, and Cluster Computing · Computer Science 2015-12-08 Songze Li , Mohammad Ali Maddah-Ali , A. Salman Avestimehr

Robust machine learning (ML) models can be developed by leveraging large volumes of data and distributing the computational tasks across numerous devices or servers. Federated learning (FL) is a technique in the realm of ML that facilitates…

To address data locality and privacy restrictions, Federated Learning (FL) has recently been adopted to fine-tune large language models (LLMs), enabling improved performance on various downstream tasks without requiring aggregated data.…

Distributed, Parallel, and Cluster Computing · Computer Science 2025-11-11 Han Liu , Ruoyao Wen , Srijith Nair , Jia Liu , Wenjing Lou , Chongjie Zhang , William Yeoh , Yevgeniy Vorobeychik , Ning Zhang

The exponential growth of data in current times and the demand to gain information and knowledge from the data present new challenges for database researchers. Known database systems and algorithms are no longer capable of effectively…

Databases · Computer Science 2017-12-06 Yaron Gonen

Citywide crowd flow analytics is of great importance to smart city efforts. It aims to model the crowd flow (e.g., inflow and outflow) of each region in a city based on historical observations. Nowadays, Convolutional Neural Networks (CNNs)…

Computer Vision and Pattern Recognition · Computer Science 2020-06-23 Yuxuan Liang , Kun Ouyang , Yiwei Wang , Ye Liu , Junbo Zhang , Yu Zheng , David S. Rosenblum

This paper investigates the role of communication in improving coordination within robot swarms, focusing on a paradigm where learning and execution occur simultaneously in a decentralized manner. We highlight the role communication can…