Related papers: MR4MR: Mixed Reality for Melody Reincarnation
The advent of increasingly-growing virtual realities poses unprecedented opportunities and challenges to different societies. Artistic collectives are not an exception, and we here aim to put special attention into musicians. Compositions,…
There is increased interest in using generative AI to create 3D spaces for Virtual Reality (VR) applications. However, today's models produce artificial environments, falling short of supporting collaborative tasks that benefit from…
In virtual reality environments, the alignment of perceptual modalities is crucial for immersion and presence. In the AR domain, it is difficult to create such alignments because elements in the physical world are often beyond the user's…
The growing spread of robots for service and industrial purposes calls for versatile, intuitive and portable interaction approaches. In particular, in industrial environments, operators should be able to interact with robots in a fast,…
In Extended Reality (XR), complex acoustic environments often overwhelm users, compromising both scene awareness and social engagement due to entangled sound sources. We introduce MoXaRt, a real-time XR system that uses audio-visual cues to…
Recent years have seen immense progress in 3D computer vision and computer graphics, with emerging tools that can virtualize real-world 3D environments for numerous Mixed Reality (XR) applications. However, alongside immersive visual…
Mixed Reality (MR) technologies are increasingly being used to enrich exhibitions and public spaces by blending digital content with the physical environment in real time. However, little is known about curatorial strategies for embedding…
In the realm of music therapy, Virtual Reality (VR) has a long-standing history of enriching human experiences through immersive applications, spanning entertainment games, serious games, and professional training in various fields.…
Optical see-though head-mounted displays (OST HMDs) are one of the key technologies for merging virtual objects and physical scenes to provide an immersive mixed reality (MR) environment to its user. A fundamental limitation of HMDs is,…
Humans rely on multisensory integration to perceive spatial environments, where auditory cues enable sound source localization in three-dimensional space. Despite the critical role of spatial audio in immersive technologies such as VR/AR,…
Music visualization is an important medium that enables synesthetic experiences and creative inspiration. However, previous research focused mainly on the technical and theoretical aspects, overlooking users' everyday interaction with music…
Digital prototyping and evaluation using 3D modeling and digital human models are becoming more practical for customizing products to the preference of a user. However, the 3D modeling is less accessible to casual users, and digital human…
Recent advancements in music large language models (LLMs) have significantly improved music understanding tasks, which involve the model's ability to analyze and interpret various musical elements. These improvements primarily focused on…
We present AudioMiXR, an augmented reality (AR) interface intended to assess how users manipulate virtual audio objects situated in their physical space using six degrees of freedom (6DoF) deployed on a head-mounted display (Apple Vision…
Digital technologies and tools have transformed the way we can study cultural heritage and the way we can recreate it digitally. Techniques such as laser scanning, photogrammetry, and a variety of Mixed Reality solutions have enabled…
Augmented Reality (AR) enriches a user's real environment by adding spatially aligned virtual objects (3D models, 2D textures, textual annotations, etc) by means of special display technologies. These are either worn on the body or placed…
This project brings music to sight. Music can be a visual masterpiece. Some people naturally experience a visualization of audio - a condition called synesthesia. The type of synesthesia explored is when sounds create colors in the 'mind's…
This paper introduces HoloBots, a mixed reality remote collaboration system that augments holographic telepresence with synchronized mobile robots. Beyond existing mixed reality telepresence, HoloBots lets remote users not only be visually…
With face-to-face music collaboration being severely limited during the recent pandemic, mixed reality technologies and their potential to provide musicians a feeling of "being there" with their musical partner can offer tremendous…
The process of reconstructing experiences from human brain activity offers a unique lens into how the brain interprets and represents the world. In this paper, we introduce a method for reconstructing music from brain activity, captured…