English

Density-guided Translator Boosts Synthetic-to-Real Unsupervised Domain Adaptive Segmentation of 3D Point Clouds

Computer Vision and Pattern Recognition 2024-03-28 v1 Artificial Intelligence

Abstract

3D synthetic-to-real unsupervised domain adaptive segmentation is crucial to annotating new domains. Self-training is a competitive approach for this task, but its performance is limited by different sensor sampling patterns (i.e., variations in point density) and incomplete training strategies. In this work, we propose a density-guided translator (DGT), which translates point density between domains, and integrates it into a two-stage self-training pipeline named DGT-ST. First, in contrast to existing works that simultaneously conduct data generation and feature/output alignment within unstable adversarial training, we employ the non-learnable DGT to bridge the domain gap at the input level. Second, to provide a well-initialized model for self-training, we propose a category-level adversarial network in stage one that utilizes the prototype to prevent negative transfer. Finally, by leveraging the designs above, a domain-mixed self-training method with source-aware consistency loss is proposed in stage two to narrow the domain gap further. Experiments on two synthetic-to-real segmentation tasks (SynLiDAR \rightarrow semanticKITTI and SynLiDAR \rightarrow semanticPOSS) demonstrate that DGT-ST outperforms state-of-the-art methods, achieving 9.4%\% and 4.3%\% mIoU improvements, respectively. Code is available at \url{https://github.com/yuan-zm/DGT-ST}.

Keywords

Cite

@article{arxiv.2403.18469,
  title  = {Density-guided Translator Boosts Synthetic-to-Real Unsupervised Domain Adaptive Segmentation of 3D Point Clouds},
  author = {Zhimin Yuan and Wankang Zeng and Yanfei Su and Weiquan Liu and Ming Cheng and Yulan Guo and Cheng Wang},
  journal= {arXiv preprint arXiv:2403.18469},
  year   = {2024}
}

Comments

CVPR2024