Language Model (LM)-based speech enhancement (SE) has recently emerged as a promising direction, but existing approaches predominantly rely on token-level likelihood objectives that weakly reflect human perception. This mismatch limits progress, as optimizing signal accuracy does not always improve naturalness or listening comfort. We address this gap by introducing a perceptually aligned LM-based SE approach. Our method applies Direct Preference Optimization (DPO) with UTMOS, a neural MOS predictor, as a proxy for human ratings, directly steering models toward perceptually preferred outputs. This design directly connects model training to perceptual quality and is broadly applicable within LM-based SE frameworks. On the Deep Noise Suppression Challenge 2020 test sets, our approach consistently improves speech quality metrics, achieving relative gains of up to 56%. To our knowledge, this is the first integration of perceptual feedback into LM-based SE and the first application of DPO in the SE domain, establishing a new paradigm for perceptually aligned enhancement with SE.
@article{arxiv.2507.09929,
title = {Aligning Generative Speech Enhancement with Perceptual Feedback},
author = {Haoyang Li and Nana Hou and Yuchen Hu and Jixun Yao and Sabato Marco Siniscalchi and Xuyi Zhuang and Deheng Ye and Wei Yang and Eng Siong Chng},
journal= {arXiv preprint arXiv:2507.09929},
year = {2026}
}