Related papers: Reflective Motion and a Physical Canvas: Exploring…
In face-to-face interaction, we use multiple modalities, including speech and gestures, to communicate information and resolve references to objects. However, how representational co-speech gestures refer to objects remains understudied…
Mindfulness and relaxation techniques for mental health are increasingly being explored in the human-computer interaction community. Physiological signals and their visualization have often been exploited together in a form of biofeedback…
Embodied cognition argues that intelligence arises from sensorimotor interaction rather than passive observation. It raises an intriguing question: do modern vision-language models (VLMs), trained largely in a disembodied manner, exhibit…
Composite visualization represents a widely embraced design that combines multiple visual representations to create an integrated view. However, the traditional approach of creating composite visualizations in immersive environments…
In this position paper, we aim to spark more discussions surrounding the use of empathy as the intended outcome of many studies on immersive virtual reality experiences. As a construct, empathy has many significant flaws that may lead to…
With the advancement of conversational AI, research on bodily expressions, including gestures and facial expressions, has also progressed. However, many existing studies focus on photorealistic avatars, making them unsuitable for…
In this study, we compare the virtual and real gait parameters to investigate the effect of appearances of embodied avatars and virtual reality experience on gait in physical and virtual environments. We developed a virtual environment…
Visual Grounding, also known as Referring Expression Comprehension and Phrase Grounding, aims to ground the specific region(s) within the image(s) based on the given expression text. This task simulates the common referential relationships…
We use an immersive virtual reality environment to explore the intricate social cues that underlie non-verbal communication involved in a pedestrian's crossing decision. We "hack" non-verbal communication between pedestrian and vehicle by…
There is a mismatch between psychological and computational studies on emotions. Psychological research aims at explaining and documenting internal mechanisms of these phenomena, while computational work often simplifies them into labels.…
Human communication is inherently multimodal, involving a combination of verbal and non-verbal cues such as speech, facial expressions, and body gestures. Modeling these behaviors is essential for understanding human interaction and for…
Digital communication tools are limited to visual and auditory information and lack non-verbal information such as touch, which is important for communicating intentions and emotions. In order to solve this problem, the use of haptic…
Centrality of emotion for the stories told by humans is underpinned by numerous studies in literature and psychology. The research in automatic storytelling has recently turned towards emotional storytelling, in which characters' emotions…
The reasoning capabilities of embodied agents introduce a critical, under-explored inferential privacy challenge, where the risk of an agent generate sensitive conclusions from ambient data. This capability creates a fundamental tension…
Digital note-taking is gaining popularity, offering a durable, editable, and easily indexable way of storing notes in a vectorized form, known as digital ink. However, a substantial gap remains between this way of note-taking and…
Accuracy and speed are pivotal when typing. Mixed reality typing is typically performed by typing on a midair keyboard with your index fingers. This deprives users of both the tactile feedback available on physical devices and the ability…
Human interactions are characterized by explicit as well as implicit channels of communication. While the explicit channel transmits overt messages, the implicit ones transmit hidden messages about the communicator (e.g., his/her intentions…
Mental rotation -- the ability to compare objects seen from different viewpoints -- is a fundamental example of mental simulation and spatial world modeling in humans. Here we propose a mechanistic model of human mental rotation, leveraging…
We present a tele-immersive system that enables people to interact with each other in a virtual world using body gestures in addition to verbal communication. Beyond the obvious applications, including general online conversations and…
This paper introduces embodied communication, a new wireless communication modality in which information is imprinted onto environmental states and recovered by the receiver through sensing. No dedicated communication transmitter is…