Large language models are known to contain representational redundancy across network depth, making depth pruning an effective approach for improving inference efficiency. Existing one-shot pruning methods rely on local layer importance or fixed redundancy assumptions across architectures. We propose Locality-Aware Redundancy Pruning (LoRP), a training-free one-shot depth pruning framework guided by representation locality. We show that inter-layer redundancy can be either localized or globally distributed depending on the LLM architecture. To characterize this phenomenon, we introduce Representation Locality Score (RLS), derived from global inter-layer hidden-state similarity. Using a small calibration set, LoRP computes pairwise layer similarity, clusters layers by representational similarity, and allocates pruning according to residual intra-cluster redundancy. Experiments across diverse LLM families show improvements in both perplexity and downstream task accuracy.
@article{arxiv.2605.27786,
title = {Locality-Aware Redundancy Pruning for LLM Depth Compression},
author = {Vincent-Daniel Yun and Youngrae Kim and Woosang Lim and YoungJin Heo and Minkyu Kim and Sunwoo Lee},
journal= {arXiv preprint arXiv:2605.27786},
year = {2026}
}