Related papers: Mind Your Vision: Multimodal Estimation of Refract…
Recently multimodal transformer models have gained popularity because their performance on language and vision tasks suggest they learn rich visual-linguistic representations. Focusing on zero-shot image retrieval tasks, we study three…
With stereoscopic displays, a depth sensation that is too strong could impede visual comfort and result in fatigue or pain. Electroencephalography (EEG) is a technology which records brain activity. We used it to develop a novel…
This paper investigates the feasibility of fusing two eye-centric authentication modalities-eye movements and periocular images-within a calibration-free authentication system. While each modality has independently shown promise for user…
The main challenge of Multi-Object Tracking~(MOT) lies in maintaining a continuous trajectory for each target. Existing methods often learn reliable motion patterns to match the same target between adjacent frames and discriminative…
This study used eye-tracking to capture the students' visual attention while taking the test of understanding graphs in kinematics (TUG-K). A total of N = 115 upper-secondary-level students from Germany and Switzerland took the 26-item…
Depression is a widespread mental health disorder, yet its automatic detection remains challenging. Prior work has explored unimodal and multimodal approaches, with multimodal systems showing promise by leveraging complementary signals.…
Accurate and efficient product classification is significant for E-commerce applications, as it enables various downstream tasks such as recommendation, retrieval, and pricing. Items often contain textual and visual information, and…
In this paper, we aimed at reviewing several different approaches present today in the search for more accurate diagnostic and treatment management in mental healthcare. Our focus is on mood disorders, and in particular on the major…
Background/Introduction: Concurrent electroencephalography and resting-state functional magnetic resonance imaging (rsfMRI) have been widely used for studying the (presumably) awake and alert human brain with high temporal/spatial…
Fatigue is a broad, multifactorial concept that includes the subjective perception of reduced physical and mental energy levels. It is also one of the key factors that strongly affect patients' health-related quality of life. To date, most…
Cognitive load assessment is crucial for understanding human performance in various domains. This study investigates the impact of different task conditions and time constraints on cognitive load using multiple measures, including…
Sleep is a complex physiological process evaluated through various modalities recording electrical brain, cardiac, and respiratory activities. We curate a large polysomnography dataset from over 14,000 participants comprising over 100,000…
With the rapid development of wearable cameras, a massive collection of egocentric video for first-person visual perception becomes available. Using egocentric videos to predict first-person activity faces many challenges, including limited…
In this paper we present the first results of a pilot experiment in the capture and interpretation of multimodal signals of human experts engaged in solving challenging chess problems. Our goal is to investigate the extent to which…
Counting the repetition of human exercise and physical rehabilitation is a common task in rehabilitation and exercise training. The existing vision-based repetition counting methods less emphasize the concurrent motions in the same video.…
Electroencephalography (EEG) interpretation using multimodal large language models (MLLMs) offers a novel approach for analyzing brain signals. However, the complex nature of brain activity introduces critical challenges: EEG signals…
Electroencephalography (EEG) is a widely used technique for measuring brain activity. EEG-based signals can reveal a persons emotional state, as they directly reflect activity in different brain regions. Emotion-aware systems and EEG-based…
We cast visual retrieval as a regression problem by posing triplet loss as a regression loss. This enables epistemic uncertainty estimation using dropout as a Bayesian approximation framework in retrieval. Accordingly, Monte Carlo (MC)…
Entity state tracking is a necessary component of world modeling that requires maintaining coherent representations of entities over time. Previous work has benchmarked entity tracking performance in purely text-based tasks. We introduce…
We introduce and evaluate a novel approach for detecting smooth pursuit eye movements that increases the number of distinguishable targets and is more robust against false positives. Being natural and calibration-free, Pursuits has been…