English

Semi-Supervised Semantic Depth Estimation using Symbiotic Transformer and NearFarMix Augmentation

Computer Vision and Pattern Recognition 2023-08-29 v1 Machine Learning

Abstract

In computer vision, depth estimation is crucial for domains like robotics, autonomous vehicles, augmented reality, and virtual reality. Integrating semantics with depth enhances scene understanding through reciprocal information sharing. However, the scarcity of semantic information in datasets poses challenges. Existing convolutional approaches with limited local receptive fields hinder the full utilization of the symbiotic potential between depth and semantics. This paper introduces a dataset-invariant semi-supervised strategy to address the scarcity of semantic information. It proposes the Depth Semantics Symbiosis module, leveraging the Symbiotic Transformer for achieving comprehensive mutual awareness by information exchange within both local and global contexts. Additionally, a novel augmentation, NearFarMix is introduced to combat overfitting and compensate both depth-semantic tasks by strategically merging regions from two images, generating diverse and structurally consistent samples with enhanced control. Extensive experiments on NYU-Depth-V2 and KITTI datasets demonstrate the superiority of our proposed techniques in indoor and outdoor environments.

Keywords

Cite

@article{arxiv.2308.14400,
  title  = {Semi-Supervised Semantic Depth Estimation using Symbiotic Transformer and NearFarMix Augmentation},
  author = {Md Awsafur Rahman and Shaikh Anowarul Fattah},
  journal= {arXiv preprint arXiv:2308.14400},
  year   = {2023}
}

Comments

Accepted at WACV 2024

R2 v1 2026-06-28T12:05:50.190Z