Token-Based Audio Inpainting via Discrete Diffusion
Abstract
Audio inpainting seeks to restore missing segments in degraded recordings. Previous diffusion-based methods exhibit impaired performance when the missing region is large. We introduce the first approach that applies discrete diffusion over tokenized music representations from a pre-trained audio tokenizer, enabling stable and semantically coherent restoration of long gaps. Our method further incorporates two training approaches: a derivative-based regularization loss that enforces smooth temporal dynamics, and a span-based absorbing transition that provides structured corruption during diffusion. Experiments on the MusicNet and MAESTRO datasets with gaps up to 750 ms show that our approach consistently outperforms strong baselines across range of gap lengths, for gaps of 150 ms and above. This work advances musical audio restoration and introduces new directions for discrete diffusion model training. Visit our project page for examples and code.
Cite
@article{arxiv.2507.08333,
title = {Token-Based Audio Inpainting via Discrete Diffusion},
author = {Tali Dror and Iftach Shoham and Moshe Buchris and Oren Gal and Haim Permuter and Gilad Katz and Eliya Nachmani},
journal= {arXiv preprint arXiv:2507.08333},
year = {2026}
}