DODA: Data-Oriented Sim-to-Real Domain Adaptation for 3D Semantic Segmentation

Abstract

<p>Deep learning approaches achieve prominent success in 3D semantic segmentation. However, collecting densely annotated real-world 3D datasets is extremely time-consuming and expensive. Training models on synthetic data and generalizing on real-world scenarios becomes an appealing alternative, but unfortunately suffers from notorious domain shifts. In this work, we propose a <strong>D</strong>ata-<strong>O</strong>riented <strong>D</strong>omain <strong>A</strong>daptation (DODA) framework to mitigate pattern and context gaps caused by different sensing mechanisms and layout placements across domains. Our DODA encompasses virtual scan simulation to imitate real-world point cloud patterns and tail-aware cuboid mixing to alleviate the interior context gap with a cuboid-based intermediate domain. The first unsupervised sim-to-real adaptation benchmark on 3D indoor semantic segmentation is also built on 3D-FRONT, ScanNet and S3DIS along with 8 popular Unsupervised Domain Adaptation (UDA) methods. Our DODA surpasses existing UDA approaches by over 13% on both 3D-FRONT →→ ScanNet and 3D-FRONT →→ S3DIS. Code is available at <a href="https://github.com/CVMI-Lab/DODA">https://github.com/CVMI-Lab/DODA</a>.<br></p>