Related papers: Galvanic vestibular stimulation produces cross-mod…
Robust audio-visual speech recognition (AVSR) in noisy environments remains challenging, as existing systems struggle to estimate audio reliability and dynamically adjust modality reliance. We propose router-gated cross-modal feature…
Audio-visual speech enhancement (AVSE) has been found to be particularly useful at low signal-to-noise (SNR) ratios due to the immunity of the visual features to acoustic noise. However, a significant gap exists in AVSE methods tailored to…
Stochastic Resonance (SR) describes a phenomenon where an additive noise (stochastic carrier-wave) enhances the signal transmission in a nonlinear system. In the nervous system, nonlinear properties are present from the level of single ion…
Audio-Visual Speech Recognition (AVSR) combines auditory and visual speech cues to enhance the accuracy and robustness of speech recognition systems. Recent advancements in AVSR have improved performance in noisy environments compared to…
Visual Speech Recognition (VSR) aims to recognize corresponding text by analyzing visual information from lip movements. Due to the high variability and weak information of lip movements, VSR tasks require effectively utilizing any…
Humans have the ability to utilize visual cues, such as lip movements and visual scenes, to enhance auditory perception, particularly in noisy environments. However, current Automatic Speech Recognition (ASR) or Audio-Visual Speech…
In real-world environments, background noise significantly degrades the intelligibility and clarity of human speech. Audio-visual speech enhancement (AVSE) attempts to restore speech quality, but existing methods often fall short,…
In audiovisual automatic speech recognition (AV-ASR) systems, information fusion of visual features in a pre-trained ASR has been proven as a promising method to improve noise robustness. In this work, based on the prominent Whisper ASR,…
Vision is known to improve human postural responses to external perturbations. This study investigates the role of vision for the responses to continuous pseudorandom support surface translations in the body sagittal plane in three visual…
Novel view acoustic synthesis (NVAS) aims to render binaural audio at any target viewpoint, given a mono audio emitted by a sound source at a 3D scene. Existing methods have proposed NeRF-based implicit models to exploit visual cues as a…
Human speech processing is inherently multimodal, where visual cues (lip movements) help to better understand the speech in noise. Lip-reading driven speech enhancement significantly outperforms benchmark audio-only approaches at low…
Background noise reduces speech intelligibility and quality, making speaker verification (SV) in noisy environments a challenging task. To improve the noise robustness of SV systems, additive noise data augmentation method has been commonly…
The main objective of this work is to explore aspects of stochastic resonance (SR) in noisy bistable, symmetric systems driven by subthreshold periodic rectangular external signals possessing a large duty cycle of unity. Using a precise…
Adding noise to a sensory signal generally decreases human performance. However noise can improve performance too, due to a process called stochastic resonance (SR). This paradoxical effect may be exploited in psychophysical experiments, to…
Non-intrusive assessment of speech quality and intelligibility is essential when clean reference signals are unavailable. In this work, we propose a multimodal framework that integrates audio features and visual cues to predict PESQ and…
Neuronal membrane potentials fluctuate stochastically due to conductance changes caused by random transitions between the open and close states of ion channels. Although it has previously been shown that channel noise can nontrivially…
We investigate novel approach, which improves the sensitivity of gravitational wave (GW) interferometer due to stochastic resonance (SR) phenomenon, performing in additional nonlinear cavity (NC). The NC is installed in the output of…
Visual neural decoding aims to extract and interpret original visual experiences directly from human brain activity. Recent studies have demonstrated the feasibility of decoding visual semantic categories from electroencephalography (EEG)…
Gyrosonics refers to novel audio binaural stimulus that produces rotational perceptions of sound movement in head at a particular predetermined frequency. Therapeutic effect observed with this is considered to be associated with…
Audio-visual speech recognition (AVSR) aims to transcribe human speech using both audio and video modalities. In practical environments with noise-corrupted audio, the role of video information becomes crucial. However, prior works have…