Related papers: SkyMemory: A LEO Edge Cache for Transformer Infere…

LEO satellite constellation coverage has served as the base platform for various space applications. However, the rapidly evolving security environment such as orbit debris and adversarial space threats are greatly endangering the security…

In recommendation systems, practitioners observed that increase in the number of embedding tables and their sizes often leads to significant improvement in model performances. Given this and the business importance of these models to major…

In this paper, we propose scalable distributed beamforming schemes over low Earth orbit (LEO) satellite networks that rely solely on statistical channel state information for downlink orthogonal frequency division multiplexing systems. We…

Modern storage systems predominantly use flash-based SSDs as a cache layer due to their favorable performance and cost efficiency. However, in tiny-object workloads, existing flash cache designs still suffer from high write amplification.…

Large language models encounter critical GPU memory capacity constraints during long-context inference, where KV cache memory consumption severely limits decode batch sizes. While existing research has explored offloading KV cache to DRAM,…

Satellite missions demand ever greater connectivity, especially in the LEO regime. In this paper, we introduce the new mega-constellation services in space paradigm: we show that megaconstellations, deployed to offer innovative services to…

It is widely acknowledged that the performance of Transformer models is logarithmically related to their number of parameters and computational complexity. While approaches like Mixture of Experts (MoE) decouple parameter count from…

Large Language Model (LLM) inference on large-scale systems is expected to dominate future cloud infrastructures. Efficient LLM inference in cloud environments with numerous AI accelerators is challenging, necessitating extensive…

In satellite constellation-based communication systems, continuous user coverage requires frequent handoffs due to the dynamic topology induced by the Low Earth Orbit (LEO) satellites. Each handoff between a satellite and ground users…

Key-Value (KV) cache plays a crucial role in accelerating inference in large language models (LLMs) by storing intermediate attention states and avoiding redundant computation during autoregressive generation. However, its memory footprint…

Earth Observation (EO) has traditionally involved the transmission of a large volume of raw data to map the Earth surface. This results in congestion to the satellite network and delays in the availability of the results, invalidating the…

The increasing adoption of large language models (LLMs) necessitates inference serving systems that can deliver both high throughput and low latency. Deploying LLMs with hundreds of billions of parameters on memory-constrained GPUs exposes…

The emergence of Low Earth Orbit (LEO) satellite constellations dedicated to positioning applications holds the promise of improving the capabilities of existing Global Navigation Satellite Systems (GNSS). However, the absence of…

The use of Low Earth Orbit (LEO) satellites in the next generation (Next-G) communication systems has been gaining traction over the last few years due to their potential for providing global connectivity with low latency. Since they are…

Long-horizon LLM inference turns the key--value (KV) cache into the dominant GPU memory consumer and makes per-token attention increasingly expensive. Many common eviction policies use static recency windows or historical attention, leaving…

Serving Large Language Models (LLMs) at scale requires meeting strict Service Level Objectives (SLOs) under severe computational and memory constraints. Nevertheless, traditional caching strategies fall short: exact-matching and prefix…

Extended reality-enabled Internet of Things (XRI) provides the new user experience and the sense of immersion by adding virtual elements to the real world through Internet of Things (IoT) devices and emerging 6G technologies. However, the…

Huge memory consumption has been a major bottleneck for deploying high-throughput large language models in real-world applications. In addition to the large number of parameters, the key-value (KV) cache for the attention mechanism in the…

Serving large language models (LLMs) is important for cloud providers, and caching intermediate results (KV\$) after processing each request substantially improves serving throughput and latency. However, there is limited understanding of…

Multimodal large language model (MLLM) inference splits into two phases with opposing hardware demands: vision encoding is compute-bound, while language generation is memory-bandwidth-bound. We show that under standard transformer KV…