We present Y-MAP-Net, a Y-shaped neural network architecture designed for real-time multi-task learning on RGB images. Y-MAP-Net, simultaneously predicts depth, surface normals, human pose, semantic segmentation and generates multi-label captions, all from a single network evaluation. To achieve this, we adopt a multi-teacher, single-student training paradigm, where task-specific foundation models supervise the network's learning, enabling it to distill their capabilities into a lightweight architecture suitable for real-time applications. Y-MAP-Net, exhibits strong generalization, simplicity and computational efficiency, making it ideal for robotics and other practical scenarios. To support future research, we will release our code publicly.
@article{arxiv.2411.10334,
title = {Y-MAP-Net: Real-time depth, normals, segmentation, multi-label captioning and 2D human pose in RGB images},
author = {Ammar Qammaz and Nikolaos Vasilikopoulos and Iason Oikonomidis and Antonis A. Argyros},
journal= {arXiv preprint arXiv:2411.10334},
year = {2024}
}