While beam search improves speech recognition quality over greedy decoding, standard implementations are slow, often sequential, and CPU-bound. To fully leverage modern hardware capabilities, we present a novel open-source FlexCTC toolkit for fully GPU-based beam decoding, designed for Connectionist Temporal Classification (CTC) models. Developed entirely in Python and PyTorch, it offers a fast, user-friendly, and extensible alternative to traditional C++, CUDA, or WFST-based decoders. The toolkit features a high-performance, fully batched GPU implementation with eliminated CPU-GPU synchronization and minimized kernel launch overhead via CUDA Graphs. It also supports advanced contextualization techniques, including GPU-powered N-gram language model fusion and phrase-level boosting. These features enable accurate and efficient decoding, making them suitable for both research and production use.
@article{arxiv.2508.07315,
title = {FlexCTC: GPU-powered CTC Beam Decoding With Advanced Contextual Abilities},
author = {Lilit Grigoryan and Vladimir Bataev and Nikolay Karpov and Andrei Andrusenko and Vitaly Lavrukhin and Boris Ginsburg},
journal= {arXiv preprint arXiv:2508.07315},
year = {2025}
}
Comments
Accepted to Automatic Speech Recognition and Understanding Workshop (ASRU) 2025