CIRKDV2: Cross-Image Relational Knowledge Distillation with Contextual Modeling for Efficient Semantic Segmentation

This repository contains the source code of CIRKDV2 and implementations of semantic segmentation distillation methods on popular datasets.

Requirement

Ubuntu 22.04 LTS

Python 3.9 (Anaconda is recommended)

CUDA 12.4

Install python packages:

pip install -r requirements.txt

Backbones pretrained on ImageNet-1K:

| Type | Backbone | Pretrained | | -- | -- | -- | | CNN | ResNet-101 |Download| | CNN | ResNet-18 | Download | | CNN | MobileNetV3-Small | Download | | CNN | MobileNetV3-Large | Download | | Transformer | MobileViT-XXS | Download | | Transformer | MiT-B0 | Download | | Transformer | MiT-B4 | Download |

Supported datasets:

| Dataset | Train Size | Val Size | Test Size | Class | Link | | -- | -- | -- |-- |-- |-- | | Cityscapes | 2975 | 500 | 1525 |19| Download| | Pascal VOC Aug | 10582 | 1449 | -- | 21 |Download | | CamVid | 367 | 101 | 233 | 11 | Download| | ADE20K | 20210 | 2000 | -- | 150 | Download| | COCO-Stuff-164K | 118287 | 5000 |-- | 182 | Download|

Distillation performance on Cityscapes

| Role | Network |Method | Test mIoU | Pretrained | Script | | -- | -- | -- |-- |-- |-- | | Teacher | DeepLabV3-ResNet101| -|78.30 | Download | -| | Student | DeepLabV3-ResNet18| Baseline | 73.56 | -| Train|Eval| | Student | DeepLabV3-ResNet18| CIRKDV2 | 75.60| Download| Train|Eval| | Student | UperNet-ResNet18| Baseline | 68.90 |- | Train|Eval| | Student | UperNet-ResNet18| CIRKDV2 | 72.11| Download| Train|Eval| | Student | DeepLabV3-MobileNetV3-Small| Baseline | 65.05 |- | Train|Eval| | Student | DeepLabV3-MobileNetV3-Small| CIRKDV2 | 67.62|Download | Train|Eval| | Student | PSPNet-MobileNetV3-Small| Baseline | 62.78 | -| Train|Eval| | Student | PSPNet-MobileNetV3-Small| CIRKDV2 | 65.42| Download| Train|Eval| | Student | DeepLabV3-MobileViT-XXS| Baseline | 66.24 | -| Train|Eval| | Student | DeepLabV3-MobileViT-XXS| CIRKDV2 | 68.91| Download|Train|Eval | | Student | PSPNet-MobileViT-XXS| Baseline | 65.48 | -| Train|Eval| | Student | PSPNet-MobileViT-XXS| CIRKDV2 | 68.45| Download| Train|Eval|

| Role | Network |Method | Test mIoU | Pretrained | Script | | -- | -- | -- |-- |-- |-- | | Teacher | SegFormer-MiT-B4| -| 80.38 |Download |- | | Student | SegFormer-MiT-B0| Baseline | 74.12 | -|Train|Eval | | Student |SegFormer-MiT-B0 | CIRKDV2 | 75.52|Download |Train|Eval |

You can zip the resulting images and submit it to the Cityscapes test server to obtain the test mIoU.

Distillation performance on ADE20K

| Role | Network |Method | Val mIoU | Pretrained | Script | | -- | -- | -- |-- |-- |-- | | Teacher | DeepLabV3-ResNet101| -| 43.83 |Download | | | Student | DeepLabV3-ResNet18| Baseline | 36.92 |- | Train|Eval| | Student | DeepLabV3-ResNet18| CIRKDV2 | 39.82| Download|Train|Eval | | Student | UperNet-ResNet-18| Baseline | 34.37 |- | Train|Eval| | Student | UperNet-ResNet-18| CIRKDV2 | 36.87| Download|Train|Eval | | Student | DeepLabV3-MobileNetV3-Large| Baseline | 32.83 | -|Train|Eval | | Student | DeepLabV3-MobileNetV3-Large| CIRKDV2 | 36.14| Download| Train|Eval| | Student |PSPNet-MobileNetV3-Large | Baseline | 33.63 |- | Train|Eval| | S