FastFCN: Rethinking Dilated Convolution in the Backbone for Semantic Segmentation

[Project] [Paper] [arXiv] [Home]

Official implementation of FastFCN: Rethinking Dilated Convolution in the Backbone for Semantic Segmentation.
A Faster, Stronger and Lighter framework for semantic segmentation, achieving the state-of-the-art performance and more than 3x acceleration.

@inproceedings{wu2019fastfcn,
  title     = {FastFCN: Rethinking Dilated Convolution in the Backbone for Semantic Segmentation},
  author    = {Wu, Huikai and Zhang, Junge and Huang, Kaiqi and Liang, Kongming and Yu Yizhou},
  booktitle = {arXiv preprint arXiv:1903.11816},
  year = {2019}
}

Contact: Hui-Kai Wu (huikaiwu@icloud.com)

Update

2020-04-15: Now support inference on a single image !!!

CUDA_VISIBLE_DEVICES=0,1,2,3 python -m experiments.segmentation.test_single_image --dataset [pcontext|ade20k] \
    --model [encnet|deeplab|psp] --jpu [JPU|JPU_X] \
    --backbone [resnet50|resnet101] [--ms] --resume {MODEL} --input-path {INPUT} --save-path {OUTPUT}

2020-04-15: New joint upsampling module is now available !!!

• --jpu [JPU|JPU_X]: JPU is the original module in the arXiv paper; JPU_X is a pyramid version of JPU.

2020-02-20: FastFCN can now run on every OS with PyTorch>=1.1.0 and Python==3.*.*

• Replace all C/C++ extensions with pure python extensions.

Version

1. Original code, producing the results reported in the arXiv paper. [branch:v1.0.0]
2. Pure PyTorch code, with torch.nn.DistributedDataParallel and torch.nn.SyncBatchNorm. [branch:latest]
3. Pure Python code. [branch:master]

Overview

Framework

Joint Pyramid Upsampling (JPU)

Install

1. PyTorch >= 1.1.0 (Note: The code is test in the environment with python=3.6, cuda=9.0)
2. Download FastFCN

   git clone https://github.com/wuhuikai/FastFCN.git
   cd FastFCN
   

3. Install Requirements

   nose
   tqdm
   scipy
   cython
   requests
   

Train and Test

PContext

python -m scripts.prepare_pcontext

| Method | Backbone | mIoU | FPS | Model | Scripts | |:----|:----|:---:|:---:|:---:|:---:| | EncNet | ResNet-50 | 49.91 | 18.77 | | | | EncNet+JPU (ours) | ResNet-50 | 51.05 | 37.56 | GoogleDrive | bash | | PSP | ResNet-50 | 50.58 | 18.08 | | | | PSP+JPU (ours) | ResNet-50 | 50.89 | 28.48 | GoogleDrive | bash | | DeepLabV3 | ResNet-50 | 49.19 | 15.99 | | | | DeepLabV3+JPU (ours) | ResNet-50 | 50.07 | 20.67 | GoogleDrive | bash | | EncNet | ResNet-101 | 52.60 (MS) | 10.51 | | | | EncNet+JPU (ours) | ResNet-101 | 54.03 (MS) | 32.02 | GoogleDrive | bash |

ADE20K

python -m scripts.prepare_ade20k

Training Set

| Method | Backbone | mIoU (MS) | Model | Scripts | |:----|:----|:---:|:---:|:---:| | EncNet | ResNet-50 | 41.11 | | | | EncNet+JPU (ours) | ResNet-50 | 42.75 | GoogleDrive | bash | | EncNet | ResNet-101 | 44.65 | | | | EncNet+JPU (ours) | ResNet-101 | 44.34 | GoogleDrive | bash |

Training Set + Val Set

| Method | Backbone | FinalScore (MS) | Model | Scripts | |:----|:----|:---:|:---:|:---:| | EncNet+JPU (ours) | ResNet-50 | | GoogleDrive | bash | | EncNet | ResNet-101 | 55.67 | | | | EncNet+JPU (ours) | ResNet-101 | 55.84 | GoogleDrive | bash |

Note: EncNet (ResNet-101) is trained with crop_size=576, while EncNet+JPU (ResNet-101) is trained with crop_size=480 for fitting 4 images into a 12G GPU.

Visual Results

|Dataset|Input|GT|EncNet|Ours| |:----|:---:|:---:|:---:|:---:| |PContext||||| |ADE20K|||||

More Visual Results

Acknowledgement

Code borrows heavily from PyTorch-Encoding.