Related papers: Virtual-Link: A Scalable Multi-Producer, Multi-Con…
Distributed inference of large language models (LLMs) using tensor parallelism can introduce communication overheads of $20$% even over GPUs connected via NVLink, a high-speed GPU interconnect. Several techniques have been proposed to…
Several novel industrial applications involve human control of vehicles, cranes, or mobile robots through various high-throughput feedback systems, such as Virtual Reality (VR) and tactile/haptic signals. The near real-time interaction…
As large language models (LLMs) continue to scale, multi-node deployment has become a necessity. Consequently, communication has become a critical performance bottleneck. Current intra-node communication libraries, like NCCL, typically make…
Visible Light Ad Hoc Networks (LANETs) is being perceived as an emerging technology to complement Radio Frequency (RF) based ad hoc networks to reduce congestion in the overloaded RF spectrum. LANET is intended to support scenarios…
Vision-and-Language Navigation (VLN) in real-world settings requires agents to process continuous visual streams and generate actions with low latency grounded in language instructions. While Video-based Large Language Models (Video-LLMs)…
High-level applications, such as machine learning, are evolving from simple models based on multilayer perceptrons for simple image recognition to much deeper and more complex neural networks for self-driving vehicle control systems.The…
Efficient parallelization of Large Language Models (LLMs) with long sequences is essential but challenging due to their significant computational and memory demands, particularly stemming from communication bottlenecks in attention…
Visible Light Communication (VLC) is a current technology which allows data to be transmitted by modulating information onto a light source. It has many advantages over traditional radio frequency communication and up to 10,000 times larger…
Modern large language models (LLMs) drive interactive AI systems but are bottlenecked by the memory-heavy growth of key-value (KV) caches, which limits real-time throughput under concurrent loads. Existing KV-cache compression methods rely…
Large deep learning models have achieved state-of-the-art performance in a wide range of tasks. These models often necessitate distributed systems for efficient training and inference. The fundamental building blocks for distributed model…
This paper proposes a cascaded multiwire-power line communication (PLC)/multiple-visible light communication (VLC) system. This hybrid architecture offers low installation cost, enhanced performance, practical feasibility, and a wide range…
Modern processors are increasingly featuring multiple cores, as well as support for hardware virtualization. While these processors are common in desktop and server-class computing, they are less prevalent in embedded and real-time systems.…
Model parameter synchronization across GPUs introduces high overheads for data-parallel training at scale. Existing parameter synchronization protocols cannot effectively leverage available network resources in the face of ever increasing…
NVMe(Non-Volatile Memory Express) is an industry standard for solid-state drives (SSDs) that has been widely adopted in data centers. NVMe virtualization is crucial in cloud computing as it allows for virtualized NVMe devices to be used by…
In the modern era of radio frequency (RF) spectrum crunch, visible light communication (VLC) is a recent and promising alternative technology that operates at the visible light spectrum. Thanks to its unlicensed and large bandwidth, VLC can…
The widespread diffusion of compute-intensive edge-AI workloads and the stringent demands of modern autonomous systems require advanced heterogeneous embedded architectures. Such architectures must support high-performance and reliable…
Shared virtual memory (SVM) is key in heterogeneous systems on chip (SoCs), which combine a general-purpose host processor with a many-core accelerator, both for programmability and to avoid data duplication. However, SVM can bring a…
With the phenomenal growth of the Internet of Things (IoT), Ultra Reliable Low Latency Communications (URLLC) has potentially been the enabler to guarantee the stringent requirements on latency and reliability. However, how to achieve low…
Deploying Vision-Language Models (VLMs) on edge devices remains challenging due to their substantial computational and memory demands, which exceed the capabilities of resource-constrained embedded platforms. Conversely, fully offloading…
This paper outlines the design of `Quest-V', which is implemented as a collection of separate kernels operating together as a distributed system on a chip. Quest-V uses virtualization techniques to isolate kernels and prevent local faults…