Computation and Language · Computer Science
On Speculative Decoding for Multimodal Large Language Models
Mukul Gagrani, Raghavv Goel, Wonseok Jeon, Junyoung Park +2
2024-04-16
Distributed, Parallel, and Cluster Computing · Computer Science
SpecEE: Accelerating Large Language Model Inference with Speculative Early Exiting
Jiaming Xu, Jiayi Pan, Yongkang Zhou, Siming Chen +4
2025-04-15
Distributed, Parallel, and Cluster Computing · Computer Science
A-IO: Adaptive Inference Orchestration for Memory-Bound NPUs
Chen Zhang, Yan Ding, Haotian Wang, Chubo Liu +2
2026-04-16
Computation and Language · Computer Science
Efficient Speculative Decoding for Llama at Scale: Challenges and Solutions
Bangsheng Tang, Carl Chengyan Fu, Fei Kou, Grigory Sizov +34
2025-08-12
Computation and Language · Computer Science
Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding
Heming Xia, Zhe Yang, Qingxiu Dong, Peiyi Wang +5
2024-06-05
Distributed, Parallel, and Cluster Computing · Computer Science
SPIN: Accelerating Large Language Model Inference with Heterogeneous Speculative Models
Fahao Chen, Peng Li, Tom H. Luan, Zhou Su +1
2025-03-21
Computation and Language · Computer Science
Towards Fast Multilingual LLM Inference: Speculative Decoding and Specialized Drafters
Euiin Yi, Taehyeon Kim, Hongseok Jeung, Du-Seong Chang +1
2024-11-12
Computation and Language · Computer Science
SPEED: Speculative Pipelined Execution for Efficient Decoding
Coleman Hooper, Sehoon Kim, Hiva Mohammadzadeh, Hasan Genc +3
2024-01-04
Computation and Language · Computer Science
Scaling Up, Speeding Up: A Benchmark of Speculative Decoding for Efficient LLM Test-Time Scaling
Shengyin Sun, Yiming Li, Xing Li, Yingzhao Lian +7
2025-09-08
Artificial Intelligence · Computer Science
Fast On-device LLM Inference with NPUs
Daliang Xu, Hao Zhang, Liming Yang, Ruiqi Liu +3
2024-12-17
Machine Learning · Computer Science
Judge Decoding: Faster Speculative Sampling Requires Going Beyond Model Alignment
Gregor Bachmann, Sotiris Anagnostidis, Albert Pumarola, Markos Georgopoulos +5
2025-02-03
Computation and Language · Computer Science
PipeInfer: Accelerating LLM Inference using Asynchronous Pipelined Speculation
Branden Butler, Sixing Yu, Arya Mazaheri, Ali Jannesari
2024-11-19
Machine Learning · Computer Science
Polybasic Speculative Decoding Through a Theoretical Perspective
Ruilin Wang, Huixia Li, Yuexiao Ma, Xiawu Zheng +3
2025-10-31
Machine Learning · Computer Science
Puzzle: Distillation-Based NAS for Inference-Optimized LLMs
Akhiad Bercovich, Tomer Ronen, Talor Abramovich, Nir Ailon +22
2025-06-04
Computation and Language · Computer Science
Draft & Verify: Lossless Large Language Model Acceleration via Self-Speculative Decoding
Jun Zhang, Jue Wang, Huan Li, Lidan Shou +3
2025-02-11
Hardware Architecture · Computer Science
SpecMamba: Accelerating Mamba Inference on FPGA with Speculative Decoding
Linfeng Zhong, Songqiang Xu, Huifeng Wen, Tong Xie +3
2025-09-25
Machine Learning · Computer Science
SpecOffload: Unlocking Latent GPU Capacity for LLM Inference on Resource-Constrained Devices
Xiangwen Zhuge, Xu Shen, Zeyu Wang, Fan Dang +5
2025-05-22
Signal Processing · Electrical Eng. & Systems
SpecFed: Accelerating Federated LLM Inference with Speculative Decoding and Compressed Transmission
Ce Zheng, Xinghan Wang, Jiahong Ning, Yuxuan Shi +2
2026-04-29