Precomputing the Future-Offset Average in TriAttention
Abstract
TriAttention is a recent method for shrinking the KV cache of long-reasoning LLMs: it scores each cached key by how much attention it is likely to receive and evicts the lowest-scoring ones. Because a key does not know how far away its future queries will sit, the score is averaged over a ladder of 17 possible future distances. We point out that this average is free: the future distance enters the score only through the position-dependent rotation, so the whole 17-fold average collapses--exactly, by a one-line algebraic identity--into a single per-band weight that is computed once, offline. Scoring a key then costs one evaluation instead of seventeen, with no change to which keys get pruned. The saving is modest and lives entirely in TriAttention's pruning-score computation, not in the attention kernel; we present it as a small, exact complement to their method, and we confirm the identity numerically.
Cite
@article{arxiv.2607.13051,
title = {Precomputing the Future-Offset Average in TriAttention},
author = {Amarnath Mukherjee},
journal= {arXiv preprint arXiv:2607.13051},
year = {2026}
}
Comments
8 pages, 2 figures