Facial behavior analysis is a broad topic with various categories such as facial emotion recognition, age, and gender recognition. Many studies focus on individual tasks while the multi-task learning approach is still an open research issue and requires more research. In this paper, we present our solution and experiment result for the Multi-Task Learning challenge of the Affective Behavior Analysis in-the-wild competition. The challenge is a combination of three tasks: action unit detection, facial expression recognition, and valance-arousal estimation. To address this challenge, we introduce a cross-attentive module to improve multi-task learning performance. Additionally, a facial graph is applied to capture the association among action units. As a result, we achieve the evaluation measure of 128.8 on the validation data provided by the organizers, which outperforms the baseline result of 30.
@article{arxiv.2207.10293,
title = {Affective Behavior Analysis using Action Unit Relation Graph and Multi-task Cross Attention},
author = {Dang-Khanh Nguyen and Sudarshan Pant and Ngoc-Huynh Ho and Guee-Sang Lee and Soo-Huyng Kim and Hyung-Jeong Yang},
journal= {arXiv preprint arXiv:2207.10293},
year = {2022}
}