Related papers: SkyMemory: A LEO Edge Cache for Transformer Infere…
Large language models (LLMs) have demonstrated transformative capabilities across diverse artificial intelligence applications, yet their deployment is hindered by substantial memory and computational demands, especially in…
In recent years, LEO satellite constellations have become envisioned as a core component of the next-generation wireless communication networks. The successive establishment of mega satellite constellations has triggered further demands for…
The increasing adoption of large language models (LLMs) with extended context windows necessitates efficient Key-Value Cache (KVC) management to optimize inference performance. Inference workloads like Retrieval-Augmented Generation (RAG)…
Emerging intelligent service scenarios in 6G communication impose stringent requirements for low latency, high reliability, and privacy preservation. Generative large language models (LLMs) are gradually becoming key enablers for the…
As private space companies such as SpaceX and Telesat are building large LEO satellite constellations to provide global broadband Internet access, researchers have proposed to embed compute services within satellite constellations to…
Transformer based Large Language Models (LLMs) have been widely used in many fields, and the efficiency of LLM inference becomes hot topic in real applications. However, LLMs are usually complicatedly designed in model structure with…
The key-value (KV) cache is a foundational optimization in Transformer-based large language models (LLMs), eliminating redundant recomputation of past token representations during autoregressive generation. However, its memory footprint…
Recent large language models (LLMs) with enormous model sizes use many GPUs to meet memory capacity requirements incurring substantial costs for token generation. To provide cost-effective LLM inference with relaxed latency constraints,…
The high mobility of satellites in Low Earth Orbit (LEO) mega-constellations induces a highly dynamic network topology, leading to many problems like frequent service disruptions. To mitigate this, Packet-based Load Balancing (PBLB) is…
KV cache has traditionally been stored in GPU memory to accelerate the decoding phase of large language model (LLM) inference. However, it is increasingly necessary to move KV caches outside GPU devices, to enable cache reuse across…
Low latency and high synchronization among users are critical for emerging multi-user virtual interaction applications. However, the existing ground-based cloud solutions are naturally limited by the complex ground topology and fiber…
Low Earth Orbit (LEO) satellite constellations are quickly being recognized as an upcoming extension of the Edge-Cloud Continuum into a 3D Continuum. Low-latency connectivity around the Earth and increasing computational power with every…
Low earth orbit (LEO) satellite constellation-enabled communication networks are expected to be an important part of many Internet of Things (IoT) deployments due to their unique advantage of providing seamless global coverage. In this…
LLMs encounter significant challenges in resource consumption nowadays, especially with long contexts. Despite extensive efforts dedicate to enhancing inference efficiency, these methods primarily exploit internal sparsity within the…
Low Earth orbit (LEO) satellites have been envisioned as a significant component of the sixth generation (6G) network architecture for achieving ubiquitous coverage and seamless access. However, the implementation of LEO satellites is…
The regenerative capabilities of next-generation satellite systems offer a novel approach to design low earth orbit (LEO) satellite communication systems, enabling full flexibility in bandwidth and spot beam management, power control, and…
In the evolving landscape of satellite communications, the deployment of Low-Earth Orbit (LEO) satellite constellations promises to revolutionize global Internet access by providing low-latency, high-bandwidth connectivity to underserved…
Low earth orbit (LEO) mega-constellations, integrating government space systems and commercial practices, have emerged as enabling technologies for the sixth generation (6G) networks due to their good merits of global coverage and…
Low Earth orbit (LEO) satellite constellations play a pivotal role in sixth-generation (6G) wireless networks by providing global coverage, massive connections, and huge capacity. In this paper, we present a novel LEO satellite…
The ever-increasing demand for ubiquitous, continuous, and high-quality services poses a great challenge to the traditional terrestrial network. To mitigate this problem, the mobile-edge-computing-enhanced low earth orbit (LEO) satellite…