Related papers: Timestamp-independent Haptic-Visual Synchronizatio…
The event camera is a novel bio-inspired vision sensor. When the brightness change exceeds the preset threshold, the sensor generates events asynchronously. The number of valid events directly affects the performance of event-based tasks,…
Mid-air haptic interfaces have several advantages - the haptic information is delivered directly to the user, in a manner that is unobtrusive to the immediate environment. They operate at a distance, thus easier to discover; they are more…
Audio-visual Navigation refers to an agent utilizing visual and auditory information in complex 3D environments to accomplish target localization and path planning, thereby achieving autonomous navigation. The core challenge of this task…
Text-to-video and image-to-video generation have made rapid progress in visual quality, but they remain limited in controlling the precise timing of motion. In contrast, audio provides temporal cues aligned with video motion, making it a…
Stereo matching and flow estimation are two essential tasks for scene understanding, spatially in 3D and temporally in motion. Existing approaches have been focused on the unsupervised setting due to the limited resource to obtain the…
By leveraging temporal dependency in video sequences, multi-frame human pose estimation algorithms have demonstrated remarkable results in complicated situations, such as occlusion, motion blur, and video defocus. These algorithms are…
Motivated by the emerging area of graph signal processing (GSP), we introduce a novel method to draw inference from spatiotemporal signals. Data acquisition in different locations over time is common in sensor networks, for diverse…
This paper studies the performance of a transmission and reception scheme for massive access under some practical challenges. One challenge is the near-far problem, i.e., an access point often receives signals from different transmitting…
Self-supervised audio-visual learning aims to capture useful representations of video by leveraging correspondences between visual and audio inputs. Existing approaches have focused primarily on matching semantic information between the…
Event camera is an asynchronous, high frequency vision sensor with low power consumption, which is suitable for human action understanding task. It is vital to encode the spatial-temporal information of event data properly and use standard…
Haptic interfaces have untapped the sense of touch to assist multimodal music learning. We have recently seen various improvements of interface design on tactile feedback and force guidance aiming to make instrument learning more effective.…
Text-to-image generation models have made significant progress in producing high-quality images from textual descriptions, yet they continue to struggle with maintaining subject consistency across multiple images, a fundamental requirement…
Telerobotics has shown promise in helping workers safely manipulate building components on construction sites; however, its primary reliance on visual feedback limits efficiency in situations with stiff contact or poor visibility. Reliable…
Recent progress in deep learning has enabled many advances in sound separation and visual scene understanding. However, extracting sound sources which are apparent in natural videos remains an open problem. In this work, we present…
Audiovisual synchronisation is the task of determining the time offset between speech audio and a video recording of the articulators. In child speech therapy, audio and ultrasound videos of the tongue are captured using instruments which…
Pseudo-haptics exploit carefully crafted visual or auditory cues to trick the brain into "feeling" forces that are never physically applied, offering a low-cost alternative to traditional haptic hardware. Here, we present a comparative…
Identifying the start time of a sequence of symbols received at the receiver, commonly referred to as \emph{frame synchronization}, is a critical task for achieving good performance in digital communications systems employing…
Recent advances in speech-aware language models have coupled strong acoustic encoders with large language models, enabling systems that move beyond transcription to produce richer outputs. Among these, word-level timestamp prediction is…
Dynamically synthesizing talking speech that actively responds to a listening head is critical during the face-to-face interaction. For example, the speaker could take advantage of the listener's facial expression to adjust the tones,…
Humans have the ability to utilize visual cues, such as lip movements and visual scenes, to enhance auditory perception, particularly in noisy environments. However, current Automatic Speech Recognition (ASR) or Audio-Visual Speech…