End-to-end transformer-based automatic speech recognition (ASR) systems often capture multiple speech traits in their learned representations that are highly entangled, leading to a lack of interpretability. In this study, we propose the explainable Disentangled-Transformer, which disentangles the internal representations into sub-embeddings with explicit content and speaker traits based on varying temporal resolutions. Experimental results show that the proposed Disentangled-Transformer produces a clear speaker identity, separated from the speech content, for speaker diarization while improving ASR performance.
@article{arxiv.2411.17846,
title = {Disentangled-Transformer: An Explainable End-to-End Automatic Speech Recognition Model with Speech Content-Context Separation},
author = {Pu Wang and Hugo Van hamme},
journal= {arXiv preprint arXiv:2411.17846},
year = {2024}
}
Comments
Accepted by the 6th IEEE International Conference on Image Processing Applications and Systems