Related papers: Exploiting Multicast for Accelerating Collective C…
Autoregressive Large Language Models (AR-LLMs) frequently exhibit implicit parallelism in sequential generation. Inspired by this, we introduce Multiverse, a new generative model that enables natively parallel generation. Multiverse…
Cross-core communication is increasingly a bottleneck as the number of processing elements increase per system-on-chip. Typical hardware solutions to cross-core communication are often inflexible; while software solutions are flexible, they…
Although large language model (LLM) based multi-agent systems (MAS) show their capability to solve complex tasks and achieve higher performance over single agent systems, they lead to huge computational overheads because of heavy…
Multicore is an integrated circuit chip that uses two or more computational engines (cores) places in a single processor. This new approach is used to split the computational work of a threaded application and spread it over multiple…
The rapid growth in the size of large language models has necessitated the partitioning of computational workloads across accelerators such as GPUs, TPUs, and NPUs. However, these parallelization strategies incur substantial data…
The surge in wireless devices and data traffic volume necessitates more efficient transmission methods. Multicasting has garnered consistent attention as a means to fulfill the increasing demand for more efficient data transmission methods.…
Multicast is a central challenge for emerging multi-hop wireless architectures such as wireless mesh networks, because of its substantial cost in terms of bandwidth. In this report, we study one specific case of multicast: broadcasting,…
The surge in popularity of short-form video content, particularly through platforms like TikTok and Instagram, has led to an exponential increase in data traffic, presenting significant challenges in network resource management. Traditional…
Multicast is the ability of a communication network to accept a single message from an application and to deliver copies of the message to multiple recipients at different location. With the development of Internet, Multicast is widely…
The rapid advancement of generative artificial intelligence has spurred innovative approaches to semantic communication, giving rise to a new paradigm known as generative semantic communication (GSC). The integration of flexible cross-modal…
Bandwidth hungry video content has become the dominant contributor to the data traffic world over. Cellular networks are constantly evolving to meet the growing traffic demands. Over the past few years, wireless multicast has been garnering…
Large inter-datacenter transfers are crucial for cloud service efficiency and are increasingly used by organizations that have dedicated wide area networks between datacenters. A recent work uses multicast forwarding trees to reduce the…
A common paradigm for scientific computing is distributed message-passing systems, and a common approach to these systems is to implement them across clusters of high-performance workstations. As multi-core architectures become increasingly…
The ever-increasing sizes of large language models necessitate distributed solutions for fast inference that exploit multi-dimensional parallelism, where computational loads are split across various accelerators such as GPU clusters.…
We present a system of session types based on adjoint logic which generalize standard binary session types. Our system allows us to uniformly capture several new behaviors in the space of asynchronous message-passing communication,…
The multicast key-dissemination problem over noiseless networks, introduced by Langberg and Effros [ITW 2022], here called the ``key-cast'' problem, captures the task of disseminating a shared secret random key to a set of terminals over a…
In this paper, we propose joint sequencing and scheduling optimization for uplink machine-type communication (MTC). We consider multiple energy-constrained MTC devices that transmit data to a base station following the time division…
Collective operations are common features of parallel programming models that are frequently used in High-Performance (HPC) and machine/ deep learning (ML/ DL) applications. In strong scaling scenarios, collective operations can negatively…
Modern multi-tenant AI clusters are increasingly communication-bound, driven by high-volume and multi-round GPU-to-GPU collective communication. Consequently, the GPU dispatcher's choice of a physical GPU subset for each tenant largely…
Generative models have achieved remarkable success across various applications, driving the demand for multi-GPU computing. Inter-GPU communication becomes a bottleneck in multi-GPU computing systems, particularly on consumer-grade GPUs. By…