English
Related papers

Related papers: Unsupervised speech enhancement with diffusion-bas…

200 papers

In this paper, we present a causal speech signal improvement system that is designed to handle different types of distortions. The method is based on a generative diffusion model which has been shown to work well in scenarios with missing…

Audio and Speech Processing · Electrical Eng. & Systems 2023-03-16 Julius Richter , Simon Welker , Jean-Marie Lemercier , Bunlong Lay , Tal Peer , Timo Gerkmann

Diffusion-based methods represented as stochastic differential equations on a continuous-time domain have recently proven successful as a non-adversarial generative model. Training such models relies on denoising score matching, which can…

Machine Learning · Computer Science 2024-11-05 Sarthak Mittal , Korbinian Abstreiter , Stefan Bauer , Bernhard Schölkopf , Arash Mehrjou

Recently, score-based generative models have been successfully employed for the task of speech enhancement. A stochastic differential equation is used to model the iterative forward process, where at each step environmental noise and white…

Audio and Speech Processing · Electrical Eng. & Systems 2023-05-31 Bunlong Lay , Simon Welker , Julius Richter , Timo Gerkmann

Denoising diffusion probabilistic models have been recently proposed to generate high-quality samples by estimating the gradient of the data density. The framework defines the prior noise as a standard Gaussian distribution, whereas the…

Machine Learning · Statistics 2022-02-22 Sang-gil Lee , Heeseung Kim , Chaehun Shin , Xu Tan , Chang Liu , Qi Meng , Tao Qin , Wei Chen , Sungroh Yoon , Tie-Yan Liu

We present SURE-Score: an approach for learning score-based generative models using training samples corrupted by additive Gaussian noise. When a large training set of clean samples is available, solving inverse problems via score-based…

Machine Learning · Computer Science 2025-04-23 Asad Aali , Marius Arvinte , Sidharth Kumar , Jonathan I. Tamir

Unsupervised speech enhancement based on variational autoencoders has shown promising performance compared with the commonly used supervised methods. This approach involves the use of a pre-trained deep speech prior along with a parametric…

Sound · Computer Science 2022-11-08 Mostafa Sadeghi , Romain Serizel

Current generative models are able to generate high-quality artefacts but have been shown to struggle with compositional reasoning, which can be defined as the ability to generate complex structures from simpler elements. In this paper, we…

Machine Learning · Computer Science 2024-08-20 Giovanni Bindi , Philippe Esling

Speech separation is a fundamental task in audio processing, typically addressed with fully supervised systems trained on paired mixtures. While effective, such systems typically rely on synthetic data pipelines, which may not reflect…

Audio and Speech Processing · Electrical Eng. & Systems 2025-09-30 Runwu Shi , Kai Li , Chang Li , Jiang Wang , Sihan Tan , Kazuhiro Nakadai

Diffusion models have recently been shown to be relevant for high-quality speech generation. Most work has been focused on generating spectrograms, and as such, they further require a subsequent model to convert the spectrogram to a…

Sound · Computer Science 2024-03-12 Roi Benita , Michael Elad , Joseph Keshet

Diffusion-based speech enhancement (SE) achieves natural-sounding speech and strong generalization, yet suffers from key limitations like generative artifacts and high inference latency. In this work, we systematically study artifact…

This paper presents a generative approach to speech enhancement based on a recurrent variational autoencoder (RVAE). The deep generative speech model is trained using clean speech signals only, and it is combined with a nonnegative matrix…

Machine Learning · Computer Science 2020-02-11 Simon Leglaive , Xavier Alameda-Pineda , Laurent Girin , Radu Horaud

Score diffusion methods can learn probability densities from samples. The score of the noise-corrupted density is estimated using a deep neural network, which is then used to iteratively transport a Gaussian white noise density to a target…

Computer Vision and Pattern Recognition · Computer Science 2024-10-16 Zahra Kadkhodaie , Stéphane Mallat , Eero P. Simoncelli

By learning the gradient of smoothed data distributions, diffusion models can iteratively generate samples from complex distributions. The learned score function enables their generalization capabilities, but how the learned score relates…

Machine Learning · Computer Science 2024-12-16 Binxu Wang , John J. Vastola

Speech enhancement (SE) is the foundational task of enhancing the clarity and quality of speech in the presence of non-stationary additive noise. While deterministic deep learning models have been commonly employed for SE, recent research…

Audio and Speech Processing · Electrical Eng. & Systems 2025-03-11 Sonal Kumar , Sreyan Ghosh , Utkarsh Tyagi , Anton Jeran Ratnarajah , Chandra Kiran Reddy Evuru , Ramani Duraiswami , Dinesh Manocha

Generative speech enhancement has recently shown promising advancements in improving speech quality in noisy environments. Multiple diffusion-based frameworks exist, each employing distinct training objectives and learning techniques. This…

Audio and Speech Processing · Electrical Eng. & Systems 2025-01-22 Julius Richter , Danilo de Oliveira , Timo Gerkmann

Diffusion model, as a new generative model which is very popular in image generation and audio synthesis, is rarely used in speech enhancement. In this paper, we use the diffusion model as a module for stochastic refinement. We propose…

Sound · Computer Science 2022-11-01 Zhibin Qiu , Mengfan Fu , Yinfeng Yu , LiLi Yin , Fuchun Sun , Hao Huang

We present in this paper an informed single-channel dereverberation method based on conditional generation with diffusion models. With knowledge of the room impulse response, the anechoic utterance is generated via reverse diffusion using a…

Audio and Speech Processing · Electrical Eng. & Systems 2023-06-22 Jean-Marie Lemercier , Simon Welker , Timo Gerkmann

Generative diffusion models have emerged as leading models in speech and image generation. However, in order to perform well with a small number of denoising steps, a costly tuning of the set of noise parameters is needed. In this work, we…

Machine Learning · Computer Science 2021-09-14 Robin San-Roman , Eliya Nachmani , Lior Wolf

Speech enhancement in hearing aids remains a difficult task in nonstationary acoustic environments, mainly because current signal processing algorithms rely on fixed, manually tuned parameters that cannot adapt in situ to different users or…

Although deep neural network (DNN)-based speech enhancement (SE) methods outperform the previous non-DNN-based ones, they often degrade the perceptual quality of generated outputs. To tackle this problem, we introduce a DNN-based generative…

Audio and Speech Processing · Electrical Eng. & Systems 2023-08-31 Ryosuke Sawata , Naoki Murata , Yuhta Takida , Toshimitsu Uesaka , Takashi Shibuya , Shusuke Takahashi , Yuki Mitsufuji