Ordering Matters: Rank-Aware Selective Fusion for Blended Emotion Recognition
Abstract
Blended emotion recognition is challenging because emotions are often expressed as mixtures of subtle and overlapping multimodal cues rather than a single dominant signal. We propose a rank-aware multi-encoder framework that selectively combines complementary representations from diverse pre-extracted video and audio encoders. Our method projects heterogeneous encoder features into a shared latent space, estimates sample-wise encoder importance through an attention-based gating module, and fuses only the top-n most informative encoders. To better model blended emotions, we decouple prediction into presence and salience heads and align them through probability-level fusion. We further incorporate feature-level unsupervised domain adaptation without pseudo-labeling to improve robustness under distribution shift. Experiments on the BlEmoRE challenge show that the proposed framework outperforms strong individual encoders and na\"ive multi-encoder fusion baselines. Our final system ranked 2nd in the competition, supporting the effectiveness of rank-aware selective fusion for fine-grained blended emotion recognition.
Keywords
Cite
@article{arxiv.2605.21417,
title = {Ordering Matters: Rank-Aware Selective Fusion for Blended Emotion Recognition},
author = {Junghyun Lee and Hyunseo Kim and Hanna Jang and Junhyug Noh},
journal= {arXiv preprint arXiv:2605.21417},
year = {2026}
}
Comments
Accepted at IEEE FG 2026 Workshops. Final system ranked 2nd in the BlEmoRE Challenge. 9 pages including appendix, 8 figures