通过联合 KV 缓存传输和 Token 预填充实现低时延边缘 LLM 接力
信号处理
2026-03-31 v1
摘要
边缘部署大型语言模型 (LLM) 可降低交互式服务的时延，但移动性会在用户设备 (UE) 在基站 (BS) 之间接力时引入服务中断。为快速恢复解码，目标侧边缘服务器必须恢复 UE 上下文状态，这可以通过令牌转发后进行预填充计算或通过后备网络直接传输键值 (KV) 缓存来实现。本文提出一种统一的接力 (HO) 设计，联合选择预填充长度并调度后备网络 KV 缓存传输，以最小化多个 UE 的最大用户 LLM 接力时延。 resulting scheme admits a tractable step-wise solution with explicit feasibility conditions and a constructive rate-scheduling policy. Simulations show that the proposed method consistently outperforms baselines across a wide range of backhaul capacities, prefill speeds, and context sizes, providing practical guidelines for mobility-aware Edge LLM token streaming.
引用
@article{arxiv.2603.28018,
title = {Low-Latency Edge LLM Handover via Joint KV Cache Transfer and Token Prefill},
author = {Seunghun Lee and Jihong Park and Ce Zheng and Hyuncheol Park},
journal= {arXiv preprint arXiv:2603.28018},
year = {2026}
}