Document parsing is a fundamental task in multimodal understanding, supporting a wide range of downstream applications such as information extraction and intelligent document analysis. Benefiting from strong semantic modeling and robust generalization, VLM-based end-to-end approaches have emerged as the mainstream paradigm in recent years. However, these models often suffer from substantial inference latency, as they must autoregressively generate long, full-page sequences when processing long-form documents. While recent hybrid methods mitigate this issue via region-level parallel decoding with VLMs, independent region decoding loses full-page context and might weaken global coherence. To address this issue, we propose Hierarchical Speculative Decoding (HSD), a two-stage local-to-global framework for document parsing. HSD first employs a lightweight pipeline drafter to predict region partitions and generate coarse drafts for each region. The first stage verifies the generated region-level drafts in parallel for efficiency, while the second stage further performs page-level verification on these refined outputs to preserve full-page coherence. Experimental results show that our HSD achieves a 2.78x near-lossless speedup with HunyuanOCR on OmniDocBench v1.5 and up to 7.04x speedup on long-document parsing tasks, demonstrating the effectiveness of our proposed method. We will release our code to facilitate reproducibility.
@article{arxiv.2602.12957,
title = {HSD: Training-Free Acceleration for Document Parsing Vision-Language Model with Hierarchical Speculative Decoding},
author = {Wenhui Liao and Hongliang Li and Pengyu Xie and Xinyu Cai and Yufan Shen and Yi Xin and Qi Qin and Shenglong Ye and Tianbin Li and Ming Hu and Junjun He and Yihao Liu and Wenhai Wang and Min Dou and Bin Fu and Botian Shi and Yu Qiao and Lianwen Jin},
journal= {arXiv preprint arXiv:2602.12957},
year = {2026}
}