An Efficient vLLM-Based Inference Pipeline for Unified Audio Understanding and Generation

Haoran Wang; Jinchuan Tian; Siddhant Arora; Shinji Watanabe

An Efficient vLLM-Based Inference Pipeline for Unified Audio Understanding and Generation

Audio and Speech Processing 2026-07-02 v1 Artificial Intelligence

Authors: Haoran Wang , Jinchuan Tian , Siddhant Arora , Shinji Watanabe

Abstract

While Large Multimodal Models excel in comprehension, high-throughput inference engines lack native support for multimodal generation. This is severe in Speech Language Models, where generating multi-layered audio tokens via decoupled AR+NAR or synchronous Multi-Token Prediction (MTP) with delay-pattern interleaving conflicts with standard single-stream loops. We present a vLLM-based inference pipeline for unified speech understanding and generation. We extend autoregressive decoding to natively execute delay-pattern de-interleaving and coordinated multi-stream sampling, integrating an on-GPU acoustic decoder for end-to-end waveform synthesis. Crucially, we overcome the shared intuition that Classifier-Free Guidance (CFG) halves throughput. By co-scheduling paired conditional and unconditional requests within a continuous batch, our CFG implementation sustains 80% of non-CFG throughput, absorbing dual-request and logit merging overheads. We open-source our framework.

Cite

@article{arxiv.2607.02119,
  title  = {An Efficient vLLM-Based Inference Pipeline for Unified Audio Understanding and Generation},
  author = {Haoran Wang and Jinchuan Tian and Siddhant Arora and Shinji Watanabe},
  journal= {arXiv preprint arXiv:2607.02119},
  year   = {2026}
}