Semantic Scene Completion

This repo contains code for the following papers

• AICNet (CVPR2020): Anisotropic Convolutional Networks for 3D Semantic Scene Completion
• DDRNet (CVPR2019): RGBD Based Dimensional Decomposition Residual Network for 3D Semantic Scene Completion
• PALNet (RAL2019): Depth Based Semantic Scene Completion with Position Importance Aware Loss

<p align="center"> <img src='teaser_480p.gif' height="280" > </p>

Contents

0. Installation
1. Data Preparation
2. Train and Test
3. Visualization and Evaluation
4. Citation

Installation

Environment

• Ubuntu 16.04
• python 3.6
• CUDA 10.1

Requirements:

• pytorch≥1.4.0
• torch_scatter
• imageio
• scipy
• scikit-learn
• tqdm

You can install the requirements by running pip install -r requirements.txt.

If you use other versions of PyTorch or CUDA, be sure to select the corresponding version of torch_scatter.

Data Preparation

Download dataset

The raw data can be found in SSCNet.

The repackaged data can be downloaded via Google Drive or BaiduYun(Access code:lpmk).

The repackaged data includes:

rgb_tensor   = npz_file['rgb']		# pytorch tensor of color image
depth_tensor = npz_file['depth']	# pytorch tensor of depth 
tsdf_hr      = npz_file['tsdf_hr']  	# flipped TSDF, (240, 144, 240)
tsdf_lr      = npz_file['tsdf_lr']  	# flipped TSDF, ( 60,  36,  60)
target_hr    = npz_file['target_hr']	# ground truth, (240, 144, 240)
target_lr    = npz_file['target_lr']	# ground truth, ( 60,  36,  60)
position     = npz_file['position']	# 2D-3D projection mapping index

Train and Test

Configure the data path in config.py

'train': '/path/to/your/training/data'

'val': '/path/to/your/testing/data'

Train

Edit the training script run_SSC_train.sh, then run

bash run_SSC_train.sh

Test

Edit the testing script run_SSC_test.sh, then run

bash run_SSC_test.sh

Visualization and Evaluation

comging soon

Citation

If you find this work useful in your research, please cite our paper(s):

@inproceedings{Li2020aicnet,
  author     = {Jie Li, Kai Han, Peng Wang, Yu Liu, and Xia Yuan},
  title      = {Anisotropic Convolutional Networks for 3D Semantic Scene Completion},
  booktitle  = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
  year       = {2020},
}

@InProceedings{Li2019ddr,
    author    = {Li, Jie and Liu, Yu and Gong, Dong and Shi, Qinfeng and Yuan, Xia and Zhao, Chunxia and Reid, Ian},
    title     = {RGBD Based Dimensional Decomposition Residual Network for 3D Semantic Scene Completion},
    booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
    month     = {June},
    pages     = {7693--7702},
    year      = {2019}
}

@article{li2019palnet,
  title={Depth Based Semantic Scene Completion With Position Importance Aware Loss},
  author={Li, Jie and Liu, Yu and Yuan, Xia and Zhao, Chunxia and Siegwart, Roland and Reid, Ian and Cadena, Cesar},
  journal={IEEE Robotics and Automation Letters},
  volume={5},
  number={1},
  pages={219--226},
  year={2019},
  publisher={IEEE}

}