Related papers: The Case for Persistent CXL switches

Large language models (LLMs) are typically served from clusters of GPUs/NPUs that consist of large number of devices. Unfortunately, communication between these devices incurs significant overhead, increasing the inference latency and cost…

The demand for high-density data storage with ultrafast accessibility motivates the search for new memory implementations. Ideally such storage devices should be robust to input error and to unreliability of individual elements; furthermore…

Emergency communications networks require in-network intelligence for timely traffic handling under dynamic demands and runtime constraints. In these environments, packets may need different inference behaviors, and conventional model…

As continuous learning based video analytics continue to evolve, the role of efficient edge servers in efficiently managing vast and dynamic datasets is becoming increasingly crucial. Unlike their compute architecture, storage and archival…

Directory-based protocols have been the de facto solution for maintaining cache coherence in shared-memory parallel systems comprising multi/many cores, where each store instruction is eagerly made globally visible by invalidating the…

The main areas of research in VLSI system design include area, high speed, and power-efficient data route logic systems. The amount of time needed to send a carry through the adder limits the pace at which addition can occur in digital…

Modern operating systems are typically POSIX-compliant with major system calls specified decades ago. The next generation of non-volatile memory (NVM) technologies raise concerns about the efficiency of the traditional POSIX-based systems.…

Ensuring uninterrupted data flow in modern networks requires robust fault-tolerant mechanisms, especially in environments where reliability and responsiveness are critical. This paper presents the design and simulation of a fault-tolerant…

Cloud computing has emerged as a corner stone of today's computing landscape. More and more customers who outsource their infrastructure benefit from the manageability, scalability and cost saving that come with cloud computing. Those…

Recent advances in logic schemes and fabrication processes have renewed interest in using superconductor electronics for energy-efficient computing and quantum control processors. However, scalable superconducting memory still poses a…

As large language models (LLMs) continue to scale, multi-node deployment has become a necessity. Consequently, communication has become a critical performance bottleneck. Current intra-node communication libraries, like NCCL, typically make…

In the context of the upgrade of the Large Hadron Collider at CERN for high-luminosity operation, the particle detectors have to cope with much higher data rates and therefore need to upgrade their data acquisition systems. This upgrade is…

Finetuning large language models (LLMs) is essential for task adaptation, yet today's serving stacks isolate inference and finetuning on separate GPU clusters -- wasting resources and under-utilizing hardware. We introduce FlexLLM, the…

Effective attention modules have played a crucial role in the success of Transformer-based large language models (LLMs), but the quadratic time and memory complexities of these attention modules also pose a challenge when processing long…

Effective data analytics on data collected from the real world usually begins with a notoriously expensive pre-processing step of data transformation and wrangling. Programming By Example (PBE) systems have been proposed to automatically…

The semantics of HPC storage systems are defined by the consistency models to which they abide. Storage consistency models have been less studied than their counterparts in memory systems, with the exception of the POSIX standard and its…

Transformers, while powerful, suffer from quadratic computational complexity and the ever-growing Key-Value (KV) cache of the attention mechanism. This paper introduces Trellis, a novel Transformer architecture with bounded memory that…

High load latency that results from deep cache hierarchies and relatively slow main memory is an important limiter of single-thread performance. Data prefetch helps reduce this latency by fetching data up the hierarchy before it is…

Deploying large language models (LLMs) for online inference is often constrained by limited GPU memory, particularly due to the growing KV cache during auto-regressive decoding. Hybrid GPU-CPU execution has emerged as a promising solution…

Memory bandwidth is known to be a performance bottleneck for FPGA accelerators, especially when they deal with large multi-dimensional data-sets. A large body of work focuses on reducing of off-chip transfers, but few authors try to improve…