DenseMTL: Cross-task Attention Mechanism for Dense Multi-task Learning
Abstract
Multi-task learning has recently emerged as a promising solution for a comprehensive understanding of complex scenes. In addition to being memory-efficient, multi-task models, when appropriately designed, can facilitate the exchange of complementary signals across tasks. In this work, we jointly address 2D semantic segmentation and three geometry-related tasks: dense depth estimation, surface normal estimation, and edge estimation, demonstrating their benefits on both indoor and outdoor datasets. We propose a novel multi-task learning architecture that leverages pairwise cross-task exchange through correlation-guided attention and self-attention to enhance the overall representation learning for all tasks. We conduct extensive experiments across three multi-task setups, showing the advantages of our approach compared to competitive baselines in both synthetic and real-world benchmarks. Additionally, we extend our method to the novel multi-task unsupervised domain adaptation setting. Our code is available at https://github.com/cv-rits/DenseMTL
Cite
@article{arxiv.2206.08927,
title = {DenseMTL: Cross-task Attention Mechanism for Dense Multi-task Learning},
author = {Ivan Lopes and Tuan-Hung Vu and Raoul de Charette},
journal= {arXiv preprint arXiv:2206.08927},
year = {2024}
}
Comments
IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) 2023