Related papers: SkyMemory: A LEO Edge Cache for Transformer Infere…

The deployment of low earth orbit (LEO) satellites with terrestrial networks can potentially increase the efficiency and reduce the cost of relaying content from a data center to a set of edge caches hosted by 6G and beyond enabled macro…

Low Earth orbit (LEO) satellites play an essential role in intelligent Earth observation by leveraging artificial intelligence models. However, limited onboard memory and excessive inference delay prevent the practical deployment of large…

While thousands of satellites photograph Earth every day, most of that data never makes it to the ground because downlink bandwidth simply cannot keep up. Processing data in the Low Earth Orbit (LEO) zone offers promising capabilities to…

Modern satellites deployed in low Earth orbit (LEO) accommodate processing payloads that can be exploited for edge computing. Furthermore, by implementing inter-satellite links, the LEO satellites in a constellation can route the data…

Prefix KV caching has become a key mechanism in LLM serving: it reduces time to first token (TTFT) by avoiding redundant computation across requests that share a prefix (i.e., the system prompt). However, the accumulated KV cache is often…

With the advent of large LEO satellite communication networks to provide global broadband Internet access, interest in providing edge computing resources within LEO networks has emerged. The LEO Edge promises low-latency, high-bandwidth…

The efforts to replace the successful, albeit aging, TCP/IP Internet architecture with a better suited one have driving research interest to information-centric alternatives. The Named Data Networking (NDN) architecture is probably one of…

Low Earth orbit (LEO) satellites play a pivotal role in global connectivity-delivering high-speed Internet, cellular coverage, and massive IoT support. With ever-growing onboard computing and storage resources, LEO satellites herald a new…

Large Language Models(LLMs) have had a profound impact on AI applications, particularly in the domains of long-text comprehension and generation. KV Cache technology is one of the most widely used techniques in the industry. It ensures…

Efficient inference with Large Language Models (LLMs) increasingly relies on Key-Value (KV) caches to store previously computed key and value vectors at each layer. These caches are essential to minimize redundant computation during…

Large language models (LLMs) are typically served from clusters of GPUs/NPUs that consist of large number of devices. Unfortunately, communication between these devices incurs significant overhead, increasing the inference latency and cost…

The revolutionary capabilities of Large Language Models (LLMs) are attracting rapidly growing popularity and leading to soaring user requests to inference serving systems. Caching techniques, which leverage data reuse to reduce computation,…

Large Language Model (LLM) inference is increasingly constrained by memory bandwidth, with frequent access to the key-value (KV) cache dominating data movement. While attention sparsity reduces some memory traffic, the relevance of past…

Recently the generative Large Language Model (LLM) has achieved remarkable success in numerous applications. Notably its inference generates output tokens one-by-one, leading to many redundant computations. The widely-used KV-Cache…

Large Language Models (LLMs) exhibit pronounced memory-bound characteristics during inference due to High Bandwidth Memory (HBM) bandwidth constraints. In this paper, we propose an L2 Cache-oriented asynchronous KV Cache prefetching method…

In real-time and high-resolution Earth observation imagery, Low Earth Orbit (LEO) satellites capture images that are subsequently transmitted to ground to create an updated map of an area of interest. Such maps provide valuable information…

Integrated sensing and communication (ISAC) and ubiquitous connectivity are two usage scenarios of sixth generation (6G) networks. In this context, low earth orbit (LEO) satellite constellations, as an important component of 6G networks, is…

Distributed prefix caching has become a core technique for efficient LLM serving. However, for long-context requests with high cache hit ratios, retrieving reusable KVCache blocks from remote servers has emerged as a new performance…

Large Language Model (LLM) inference is increasingly constrained by GPU memory capacity rather than compute throughput, driven by growing model sizes and the linear growth of the key-value (KV) cache during autoregressive decoding. Existing…

We propose an architecture with accompanying protocol for on-board satellite data infrastructure designed for Low Earth Orbit (LEO) constellations offering communication services, such as direct-to-cell connectivity. Our design leverages…