Related papers: Real-Time Auralization for First-Person Vocal Inte…
Virtual reality (VR) is not a new technology but has been in development for decades, driven by advances in computer technology. Currently, VR technology is increasingly being used in applications to enable immersive, yet controlled…
There is a sensory gulf between the Earth that humans inhabit and the digital realms in which modern AI agents are created. To develop AI agents that can sense, think, and act as flexibly as humans in real-world settings, it is imperative…
We introduce and explore a new multimodal input representation for vision-language models: acoustic field video. Unlike conventional video (RGB with stereo/mono audio), our video stream provides a spatially grounded visualization of sound…
Augmented reality (AR) has the potential to fundamentally change how people engage with increasingly interactive urban environments. However, many challenges exist in designing and evaluating these new urban AR experiences, such as…
This paper presents an immersive application where users receive sound and visual feedbacks on their interactions with a virtual environment. In this application, the users play the part of conductors of an orchestra of factory machines…
Human communication has been profoundly changed by social media, which allows users to engage in previously unheard-of ways, such as text-based conversations, video chats, and live streaming. The digital landscape has started to change in…
With the development of deep learning and artificial intelligence, audio synthesis has a pivotal role in the area of machine learning and shows strong applicability in the industry. Meanwhile, significant efforts have been dedicated by…
Mixed reality (MR) environments offer embodied spatial interaction, providing intuitive 3D manipulation capabilities that enhance the conceptual design process. Parametric modeling, a powerful and advanced architectural design method,…
Design for Voice User Interfaces (VUIs) has become more relevant in recent years due to the enormous advances of speech technologies and their growing presence in our everyday lives. Although modern VUIs still present interaction issues,…
Signal-dependent beamformers are advantageous over signal-independent beamformers when the acoustic scenario - be it real-world or simulated - is straightforward in terms of the number of sound sources, the ambient sound field and their…
This study evaluates the usage of virtual reality (VR) technologies as a teaching tool in oral placement therapy, a subset of speech therapy. The researcher distributed instructional videos using traditional lecture and modified…
Virtual reality (VR) is increasingly used as a social platform for users to interact and build connections with one another in an immersive virtual environment. Reflecting on the empirical progress in this area of study, a comprehensive…
Simulations constitute a fundamental component of medical and nursing education and traditionally employ standardized patients (SP) and high-fidelity manikins to develop clinical reasoning and communication skills. However, these methods…
We present the early-stage design and implementation of a multimodal, real-time communication analysis system intended as a foundational interaction layer for adaptive VR training. The system integrates five parallel processing streams: (1)…
Virtual Reality (VR) users often experience postural instability, i.e., balance problems, which could be a major barrier to universal usability and accessibility for all, especially for persons with balance impairments. Prior research has…
Vision-based Interfaces (VIs) are pivotal in advancing Human-Computer Interaction (HCI), particularly in enhancing context awareness. However, there are significant opportunities for these interfaces due to rapid advancements in multimodal…
Public speaking is an essential skill in everyone's professional or academic career. Nevertheless, honing this skill is often tricky because training in front of a mirror does not give feedback or inspire the same anxiety as present-ing in…
Augmented Reality (AR) solutions are providing tools that could improve applications in the medical and industrial fields. Augmentation can provide additional information in training, visualization, and work scenarios, to increase…
World models simulate environmental dynamics to enable agents to plan and reason about future states. While existing approaches have primarily focused on visual observations, real-world perception inherently involves multiple sensory…
Virtual reality (VR) is increasingly being used as a research platform for investigating human responses to environmental variables. While VR provides tremendous advantages in terms of variable isolation and manipulation, and ease of…