English

AIVA: An AI-based Virtual Companion for Emotion-aware Interaction

Computer Vision and Pattern Recognition 2025-09-04 v1

Abstract

Recent advances in Large Language Models (LLMs) have significantly improved natural language understanding and generation, enhancing Human-Computer Interaction (HCI). However, LLMs are limited to unimodal text processing and lack the ability to interpret emotional cues from non-verbal signals, hindering more immersive and empathetic interactions. This work explores integrating multimodal sentiment perception into LLMs to create emotion-aware agents. We propose \ours, an AI-based virtual companion that captures multimodal sentiment cues, enabling emotionally aligned and animated HCI. \ours introduces a Multimodal Sentiment Perception Network (MSPN) using a cross-modal fusion transformer and supervised contrastive learning to provide emotional cues. Additionally, we develop an emotion-aware prompt engineering strategy for generating empathetic responses and integrate a Text-to-Speech (TTS) system and animated avatar module for expressive interactions. \ours provides a framework for emotion-aware agents with applications in companion robotics, social care, mental health, and human-centered AI.

Keywords

Cite

@article{arxiv.2509.03212,
  title  = {AIVA: An AI-based Virtual Companion for Emotion-aware Interaction},
  author = {Chenxi Li},
  journal= {arXiv preprint arXiv:2509.03212},
  year   = {2025}
}
R2 v1 2026-07-01T05:19:05.238Z