Real-time Audio Video Enhancement \\with a Microphone Array and Headphones
Audio and Speech Processing
2023-03-03 v1
Abstract
This paper presents a complete hardware and software pipeline for real-time speech enhancement in noisy and reverberant conditions. The device consists of a microphone array and a camera mounted on eyeglasses, connected to an embedded system that enhances speech and plays back the audio in headphones, with a latency of maximum 120 msec. The proposed approach relies on face detection, tracking and verification to enhance the speech of a target speaker using a beamformer and a postfiltering neural network. Results demonstrate the feasibility of the approach, and opens the door to the exploration and validation of a wide range of beamformer and speech enhancement methods for real-time speech enhancement.
Keywords
Cite
@article{arxiv.2303.00949,
title = {Real-time Audio Video Enhancement \\with a Microphone Array and Headphones},
author = {Jacob Kealey and Anthony Gosselin and Étienne Deshaies-Samson and Francis Cardinal and Félix Ducharme-Turcotte and Olivier Bergeron and Amélie Rioux-Joyal and Jérémy Bélec and François Grondin},
journal= {arXiv preprint arXiv:2303.00949},
year = {2023}
}
Comments
Submitted to IROS 2023