English

Enhance-then-Balance Modality Collaboration for Robust Multimodal Sentiment Analysis

Computation and Language 2026-04-21 v2

Abstract

Multimodal sentiment analysis (MSA) integrates heterogeneous text, audio, and visual signals to infer human emotions. While recent approaches leverage cross-modal complementarity, they often struggle to fully utilize weaker modalities. In practice, dominant modalities tend to overshadow non-verbal ones, inducing modality competition and limiting overall contributions. This imbalance degrades fusion performance and robustness under noisy or missing modalities. To address this, we propose a novel model, Enhance-then-Balance Modality Collaboration framework (EBMC). EBMC improves representation quality via semantic disentanglement and cross-modal enhancement, strengthening weaker modalities. To prevent dominant modalities from overwhelming others, an Energy-guided Modality Coordination mechanism achieves implicit gradient rebalancing via a differentiable equilibrium objective. Furthermore, Instance-aware Modality Trust Distillation estimates sample-level reliability to adaptively modulate fusion weights, ensuring robustness. Extensive experiments demonstrate that EBMC achieves state-of-the-art or competitive results and maintains strong performance under missing-modality settings.

Keywords

Cite

@article{arxiv.2604.12518,
  title  = {Enhance-then-Balance Modality Collaboration for Robust Multimodal Sentiment Analysis},
  author = {Kang He and Yuzhe Ding and Xinrong Wang and Fei Li and Chong Teng and Donghong Ji},
  journal= {arXiv preprint arXiv:2604.12518},
  year   = {2026}
}

Comments

Accepted by CVPR 2026

R2 v1 2026-07-01T12:08:25.795Z