Related papers: MLTCP: Congestion Control for DNN Training
Training deep neural networks (DNNs) is a major workload in datacenters today, resulting in a tremendously fast growth of energy consumption. It is important to reduce the energy consumption while completing the DL training jobs early in…
This paper introduces a comprehensive strategy that integrates traffic perimeter control with traffic signal control to alleviate congestion in an urban traffic network (UTN). The strategy is formulated as a lexicographic multi-objective…
Distributed machine learning (DML) technology makes it possible to train large neural networks in a reasonable amount of time. Meanwhile, as the computing power grows much faster than network capacity, network communication has gradually…
Convolutional networks (ConvNets) have become a popular approach to computer vision. It is important to accelerate ConvNet training, which is computationally costly. We propose a novel parallel algorithm based on decomposition into a set of…
Congestion control is a fundamental component of Internet infrastructure, and researchers have dedicated considerable effort to developing improved congestion control algorithms. However, despite extensive study, existing algorithms…
The goal of traffic management is efficiently utilizing network resources via adapting of source sending rates and routes selection. Traditionally, this problem is formulated into a utilization maximization problem. The single-path routing…
Today, considerable Internet traffic is sent from the datacenter and heads for users. The characteristics of connections served by servers in datacenters are usually diverse and varied over time, with continuous upgrades in network…
Network-assisted congestion control leveraging Explicit Congestion Notification (ECN) is an effective way to deal with congestion issues on the Internet. However, we believe that the existing ECN mechanism in the TCP/IP protocol stack may…
We propose TopoOpt, a novel direct-connect fabric for deep neural network (DNN) training workloads. TopoOpt co-optimizes the distributed training process across three dimensions: computation, communication, and network topology. We…
Stream Control Transmission Protocol (SCTP) was introduced in 2001 as a multipath variant to traditional transport protocols, i.e. Transmission Control Protocol (TCP) and User Datagram Protocol (UDP). Concurrent Multipath Transfer (CMT) has…
Deep neural networks (DNNs) have inspired new studies in myriad edge applications with robots, autonomous agents, and Internet-of-things (IoT) devices. However, performing inference of DNNs in the edge is still a severe challenge, mainly…
In this paper, we reveal the relationship between entropy rate and the congestion in complex network and solve it analytically for special cases. Finding maximizing entropy rate will lead to an improvement of traffic efficiency, we propose…
It is important to scale out deep neural network (DNN) training for reducing model training time. The high communication overhead is one of the major performance bottlenecks for distributed DNN training across multiple GPUs. Our…
The TCP congestion control protocol serves as the cornerstone of reliable internet communication. However, as new applications require more specific guarantees regarding data rate and delay, network management must adapt. Thus, service…
Multimodal Large Language Models (MLLMs) have achieved remarkable advances by integrating text, image, and audio understanding within a unified architecture. However, existing distributed training frameworks remain fundamentally data-blind:…
Network-on-Chip (NoC) congestion builds up during heavy traffic load and cripples the system performance by stalling the cores. Moreover, congestion leads to wasted link bandwidth due to blocked buffers and bouncing packets. Existing…
Full batch training of Graph Convolutional Network (GCN) models is not feasible on a single GPU for large graphs containing tens of millions of vertices or more. Recent work has shown that, for the graphs used in the machine learning…
This paper deals with congestion control in a software defined network (SDN) setting. Presently, explicit router schemes, such as Explicit Congestion Notification (ECN), work in conjunction with the TCP protocol to handle congestion in a…
Recently, much effort has been devoted by researchers from both academia and industry to develop novel congestion control methods. LearningCC is presented in this letter, in which the congestion control problem is solved by reinforce…
Deep neural network (DNN) training continues to scale rapidly in terms of model size, data volume, and sequence length, to the point where multiple machines are required to fit large models for training. Different distributed and parallel…