相关论文: Supporting Musical Practice Sessions Through HMD-B…
Music has the power to evoke intense emotional experiences and regulate the mood of an individual. With the advent of online streaming services, research in music recommendation services has seen tremendous progress. Modern methods…
Fundamental concepts of thermodynamics rely on abstract physical quantities such as energy, heat and entropy, which play an important role in the process of interpreting thermal phenomena and statistical mechanics. However, these quantities…
Head Mounted Displays (HMDs) allow users to experience virtual reality with a great level of immersion. However, even simple physical tasks like drinking a beverage can be difficult and awkward while in a virtual reality experience. We…
Virtual reality (VR) not only allows head-mounted display (HMD) users to immerse themselves in virtual worlds but also to share them with others. When designed correctly, this shared experience can be enjoyable. However, in typical…
This paper presents the concept of AI-supported Mini-Labs, combining smartphone-based experiments with multimodal large language models (MLLMs). Smartphones, with their integrated sensors and computational power, function as versatile…
People with visual impairments perceive their environment non-visually and often use AI-powered assistive tools to obtain textual descriptions of visual information. Recent large vision-language model-based AI-powered tools like Be My AI…
Detailed feedback on courses and lecture content is essential for their improvement and also serves as a tool for reflection. However, feedback methods are often only used sporadically, especially in mass courses, because collecting and…
Multimodal deep learning systems which employ multiple modalities like text, image, audio, video, etc., are showing better performance in comparison with individual modalities (i.e., unimodal) systems. Multimodal machine learning involves…
This work focuses on generating realistic, physically-based human behaviors from multi-modal inputs, which may only partially specify the desired motion. For example, the input may come from a VR controller providing arm motion and body…
Learning from multiple modalities, such as audio and video, offers opportunities for leveraging complementary information, enhancing robustness, and improving contextual understanding and performance. However, combining such modalities…
Large language models (LLMs) have demonstrated the ability to generate formative feedback and instructional hints in English, making them increasingly relevant for AI-assisted education. However, their ability to provide effective…
Quantification of real-time informal feedback delivered by an experienced surgeon to a trainee during surgery is important for skill improvements in surgical training. Such feedback in the live operating room is inherently multimodal,…
Device-guided music transfer adapts playback across unseen devices for users who lack them. Existing methods mainly focus on modifying the timbre, rhythm, harmony, or instrumentation to mimic genres or artists, overlooking the diverse…
We developed a novel virtual reality [VR] platform with 3-dimensional sounds to help improve sensory integration and visuomotor processing for postural control and fall prevention in individuals with balance problems related to sensory…
Multimodal foundation models that can holistically process text alongside images, video, audio, and other sensory modalities are increasingly used in a variety of real-world applications. However, it is challenging to characterize and study…
Interactive reinforcement learning, where humans actively assist during an agent's learning process, has the promise to alleviate the sample complexity challenges of practical algorithms. However, the inner workings and state of the robot…
A head-mounted display (HMD) could be an important component of augmented reality system. However, as the upper face region is seriously occluded by the device, the user experience could be affected in applications such as telecommunication…
Current music similarity models typically compute a single, monolithic score, entangling distinct musical dimensions like melody, rhythm, and timbre. This limits user control and interpretability, making it impossible to execute nuanced…
In a world where in-person context transitions more into remote and hybrid concepts, we should consider new concepts of interaction in health and rehabilitation and what advantages and disadvantages they bring. One of the rising topics is…
Despite the rise of digital libraries and online reading platforms, physical libraries still offer unique benefits for education and community engagement. However, due to the convenience of digital resources, physical library visits,…