Related papers: Deep Learning Based Audio-Visual Multi-Speaker DOA…

An objective understanding of media depictions, such as inclusive portrayals of how much someone is heard and seen on screen such as in film and television, requires the machines to discern automatically who, when, how, and where someone is…

Audio representation learning based on deep neural networks (DNNs) emerged as an alternative approach to hand-crafted features. For achieving high performance, DNNs often need a large amount of annotated data which can be difficult and…

Conventional direction of arrival (DOA) estimation algorithms suffer from performance degradation due to antenna pattern distortion and substantial computational complexity in real-time execution. The support vector regression (SVR)…

At hybrid analog-digital (HAD) transceiver, an improved HAD rotational invariance techniques (ESPRIT), called I-HAD-ESPRIT, is proposed to measure the direction of arrival (DOA) of desired user, where the phase ambiguity due to HAD…

This study demonstrates that a Multimodal Large Language Model (MLLM) adapted via Low-Rank Adaptation (LoRA) can perform both Automatic Pronunciation Assessment (APA) and Mispronunciation Detection and Diagnosis (MDD) simultaneously.…

In this paper, we propose a novel reduced-rank algorithm for direction of arrival (DOA) estimation based on the minimum variance (MV) power spectral evaluation. It is suitable to DOA estimation with large arrays and can be applied to…

We introduce DIVE, an end-to-end speaker diarization algorithm. Our neural algorithm presents the diarization task as an iterative process: it repeatedly builds a representation for each speaker before predicting the voice activity of each…

Subjective evaluations are critical for assessing the perceptual realism of sounds in audio-synthesis driven technologies like augmented and virtual reality. However, they are challenging to set up, fatiguing for users, and expensive. In…

In this paper, a deep learning approach is presented for direction of arrival estimation using automotive-grade ultrasonic sensors which are used for driving assistance systems such as automatic parking. A study and implementation of the…

Image-text retrieval has developed rapidly in recent years. However, it is still a challenge in remote sensing due to visual-semantic imbalance, which leads to incorrect matching of non-semantic visual and textual features. To solve this…

In multi-speaker applications is common to have pre-computed models from enrolled speakers. Using these models to identify the instances in which these speakers intervene in a recording is the task of speaker tracking. In this paper, we…

In this paper, we propose a novel end-to-end neural-network-based speaker diarization method. Unlike most existing methods, our proposed method does not have separate modules for extraction and clustering of speaker representations.…

Probabilistic Linear Discriminant Analysis (PLDA) has become state-of-the-art method for modeling $i$-vector space in speaker recognition task. However the performance degradation is observed if enrollment data size differs from one speaker…

Reverberation not only degrades the quality of speech for human perception, but also severely impacts the accuracy of automatic speech recognition. Prior work attempts to remove reverberation based on the audio modality only. Our idea is to…

Deep learning has shown a great potential for speech separation, especially for speech and non-speech separation. However, it encounters permutation problem for multi-speaker separation where both target and interference are speech.…

Deep neural networks (DNNs) have greatly benefited direction of arrival (DoA) estimation methods for speech source localization in noisy environments. However, their localization accuracy is still far from satisfactory due to the…

This paper presents a novel method for estimating the direction of arrival (DOA) for a non-uniform and sparse linear sensor array using the weighted lifted structure low-rank matrix completion. The proposed method uses a single snapshot…

Wide-band Direction of Arrival (DOA) estimation with sensor arrays is an essential task in sonar, radar, acoustics, biomedical and multimedia applications. Many state of the art wide-band DOA estimators coherently process frequency binned…

In this paper, an accurate direction-of-arrival (DOA) estimator is developed based on the real-valued singular value decomposition (SVD) of covariance matrix. Unitary transform on the complex-valued covariance matrix is first applied, and…

Active speaker detection requires a solid integration of multi-modal cues. While individual modalities can approximate a solution, accurate predictions can only be achieved by explicitly fusing the audio and visual features and modeling…