English
Related papers

Related papers: DSVAE: Interpretable Disentangled Representation f…

200 papers

While mel-spectrograms have been widely utilized as intermediate representations in zero-shot text-to-speech (TTS), their inherent redundancy leads to inefficiency in learning text-speech alignment. Compact VAE-based latent representations…

Audio and Speech Processing · Electrical Eng. & Systems 2025-12-02 Zhikang Niu , Shujie Hu , Jeongsoo Choi , Yushen Chen , Peining Chen , Pengcheng Zhu , Yunting Yang , Bowen Zhang , Jian Zhao , Chunhui Wang , Xie Chen

Audio pretrained models are widely employed to solve various tasks in speech processing, sound event detection, or music information retrieval. However, the representations learned by these models are unclear, and their analysis mainly…

For speaker recognition, it is difficult to extract an accurate speaker representation from speech because of its mixture of speaker traits and content. This paper proposes a disentanglement framework that simultaneously models speaker…

Audio and Speech Processing · Electrical Eng. & Systems 2023-11-02 Tianchi Liu , Kong Aik Lee , Qiongqiong Wang , Haizhou Li

Intelligent behaviour in the real-world requires the ability to acquire new knowledge from an ongoing sequence of experiences while preserving and reusing past knowledge. We propose a novel algorithm for unsupervised representation learning…

The use of well-disentangled representations offers many advantages for downstream tasks, e.g. an increased sample efficiency, or better interpretability. However, the quality of disentangled interpretations is often highly dependent on the…

Machine Learning · Computer Science 2023-03-03 Benjamin Estermann , Roger Wattenhofer

Generating realistic 3D Human-Human Interaction (HHI) requires coherent modeling of the physical plausibility of the agents and their interaction semantics. Existing methods compress all motion information into a single latent…

Computer Vision and Pattern Recognition · Computer Science 2026-03-03 Zichen Geng , Zeeshan Hayder , Bo Miao , Jian Liu , Wei Liu , Ajmal Mian

Multimodal deepfakes can exhibit subtle visual artifacts and cross-modal inconsistencies, which remain challenging to detect, especially when detectors are trained primarily on curated synthetic forgeries. Such synthetic dependence can…

Computer Vision and Pattern Recognition · Computer Science 2026-03-27 Sahibzada Adil Shahzad , Ammarah Hashmi , Junichi Yamagishi , Yusuke Yasuda , Yu Tsao , Chia-Wen Lin , Yan-Tsung Peng , Hsin-Min Wang

Diffusion models have recently set new benchmarks in Speech Enhancement (SE). However, most existing score-based models treat speech spectrograms merely as generic 2D images, applying uniform processing that ignores the intrinsic structural…

Sound · Computer Science 2026-02-03 Ke Xue , Rongfei Fan , Kai Li , Shanping Yu , Puning Zhao , Jianping An

One major challenge of disentanglement learning with variational autoencoders is the trade-off between disentanglement and reconstruction fidelity. Previous studies, which increase the information bottleneck during training, tend to lose…

Machine Learning · Computer Science 2023-10-05 Jiantao Wu , Shentong Mo , Xiang Yang , Muhammad Awais , Sara Atito , Xingshen Zhang , Lin Wang , Xiang Yang

Disentangled Representation Learning aims to improve the explainability of deep learning methods by training a data encoder that identifies semantically meaningful latent variables in the data generation process. Nevertheless, there is no…

Machine Learning · Computer Science 2024-10-08 Ruoyu Wang , Lina Yao

Voice User Interfaces (VUIs) are increasingly popular and built into smartphones, home assistants, and Internet of Things (IoT) devices. Despite offering an always-on convenient user experience, VUIs raise new security and privacy concerns…

Audio and Speech Processing · Electrical Eng. & Systems 2020-09-07 Ranya Aloufi , Hamed Haddadi , David Boyle

As speech generation technology advances, the risk of misuse through deepfake audio has become a pressing concern, which underscores the critical need for robust detection systems. However, many existing speech deepfake datasets are limited…

Sound · Computer Science 2025-07-30 Wen Huang , Yanmei Gu , Zhiming Wang , Huijia Zhu , Yanmin Qian

This paper describes a submission to the Environment-Aware Speech and Sound Deepfake Detection Challenge (ESDD2) 2026, which addresses component-level deepfake detection using the CompSpoofV2 dataset, where speech and environmental sounds…

Sound · Computer Science 2026-05-06 Khalid Zaman , Qixuan Huang , Muhammad Uzair , Masashi Unoki

Decoding EEG signals for imagined speech is a challenging task due to the high-dimensional nature of the data and low signal-to-noise ratio. In recent years, denoising diffusion probabilistic models (DDPMs) have emerged as promising…

Audio and Speech Processing · Electrical Eng. & Systems 2023-07-28 Soowon Kim , Young-Eun Lee , Seo-Hyun Lee , Seong-Whan Lee

Self-supervised learning in speech involves training a speech representation network on a large-scale unannotated speech corpus, and then applying the learned representations to downstream tasks. Since the majority of the downstream tasks…

Diffusion-based speech generators are ubiquitous. These methods can generate very high quality synthetic speech and several recent incidents report their malicious use. To counter such misuse, synthetic speech detectors have been developed.…

Audio and Speech Processing · Electrical Eng. & Systems 2024-10-03 Kratika Bhagtani , Amit Kumar Singh Yadav , Paolo Bestagini , Edward J. Delp

Deep generative models have been enjoying success in modeling continuous data. However it remains challenging to capture the representations for discrete structures with formal grammars and semantics, e.g., computer programs and molecular…

Machine Learning · Computer Science 2018-02-27 Hanjun Dai , Yingtao Tian , Bo Dai , Steven Skiena , Le Song

Improving controllability or the ability to manipulate one or more attributes of the generated data has become a topic of interest in the context of deep generative models of music. Recent attempts in this direction have relied on learning…

Sound · Computer Science 2021-08-04 Ashis Pati , Alexander Lerch

Recommendation models are typically trained on observational user interaction data, but the interactions between latent factors in users' decision-making processes lead to complex and entangled data. Disentangling these latent factors to…

Information Retrieval · Computer Science 2023-04-18 Siyu Wang , Xiaocong Chen , Quan Z. Sheng , Yihong Zhang , Lina Yao

Learning disentangled and interpretable representations is an important step towards accomplishing comprehensive data representations on the manifold. In this paper, we propose a novel representation learning algorithm which combines the…

Machine Learning · Computer Science 2021-07-13 Fei Ye , Adrian G. Bors