English

What Survives When You Compress a Recursive Reasoner for the Edge?

Machine Learning 2026-06-25 v1

Abstract

Recursive reasoning models can solve complex structured tasks with only a few million parameters by repeatedly updating a latent state. Deploying these models on edge hardware requires significant compression, but unlike conventional sequence models, quantization errors compound across recursive reasoning cycles rather than across output tokens. As a result, standard intuitions about compression fail to apply. In this work, we ask what survives when recursive reasoners are compressed. Across a full precision sweep, three tasks, and two recursive architectures, we find that aggressive compression preserves local prediction but destroys global reasoning: cell accuracy holds while puzzle-exact accuracy collapses to zero under naive INT4 pruning, distillation, and linear attention alike. Token-level objectives, including quantization-aware training, cannot repair it. The collapse is architectural -- it strikes MLP-mixing recursion but not attention on the same task -- and we reverse it with per-channel calibrated INT4 without retraining. We also introduce carry-trajectory fidelity, the cosine similarity to the full-precision reasoning path, as a label-free signal that predicts this damage and its recovery before a task evaluation. The combined result is a deployment recipe: flash-streamed embeddings remove a 99.4MB bottleneck, INT8 at one cycle matches full-depth accuracy at 6x fewer FLOPs (8MB SoC), and calibrated INT4 fits a 4MB microcontroller.

Keywords

Cite

@article{arxiv.2606.26488,
  title  = {What Survives When You Compress a Recursive Reasoner for the Edge?},
  author = {Pearse Jim and Steven Kolawole and Opegbemi Matthias Busoye and Glory Bagai and Virginia Smith},
  journal= {arXiv preprint arXiv:2606.26488},
  year   = {2026}
}

Comments

Preprint; in review