SEgmentation TRansformers -- SETR

SETR

Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers <br> Sixiao Zheng, Jiachen Lu, Hengshuang Zhao, Xiatian Zhu, Zekun Luo, Yabiao Wang, Yanwei Fu, Jianfeng Feng, Tao Xiang, Philip HS Torr, Li Zhang <br> CVPR 2021

Vision Transformers: From Semantic Segmentation to Dense Prediction [Springer] [arxiv] <br> Li Zhang, Jiachen Lu, Sixiao Zheng, Xinxuan Zhao, Xiatian Zhu, Yanwei Fu, Tao Xiang, Jianfeng Feng <br> IJCV 2024 July

SETR

Cityscapes

| Method | Crop Size | Batch size | iteration | set | mIoU | model | config | | --------------- | --------- | ---------- | --------- | --- | ----- | ----------------------------------------------------------------------------------------------- | ------------------------------------------------------------------ | | SETR-Naive | 768x768 | 8 | 40k | val | 77.37 | google drive | config | | SETR-Naive | 768x768 | 8 | 80k | val | 77.90 | google drive | config | | SETR-MLA | 768x768 | 8 | 40k | val | 76.65 | google drive | config | | SETR-MLA | 768x768 | 8 | 80k | val | 77.24 | google drive | config | | SETR-PUP | 768x768 | 8 | 40k | val | 78.39 | google drive | config | | SETR-PUP | 768x768 | 8 | 80k | val | 79.34 | google drive | config | | SETR-Naive-Base | 768x768 | 8 | 40k | val | 75.54 | google drive | config | | SETR-Naive-Base | 768x768 | 8 | 80k | val | 76.25 | google drive | config | | SETR-Naive-DeiT | 768x768 | 8 | 40k | val | 77.85 | google drive | config | | SETR-Naive-DeiT | 768x768 | 8 | 80k | val | 78.66 | google drive | config | | SETR-MLA-DeiT | 768x768 | 8 | 40k | val | 78.04 | google drive | config | | SETR-MLA-DeiT | 768x768 | 8 | 80k | val | 78.98 | google drive | config | | SETR-PUP-DeiT | 768x768 | 8 | 40k | val | 78.79 | google drive | config | | SETR-PUP-DeiT | 768x768 | 8 | 80k | val | 79.45 | google drive | config |

ADE20K

| Method | Crop Size | Batch size | iteration | set | mIoU | mIoU(ms+flip) | model | Config | | ------------- | --------- | ---------- | --------- | --- | ----- | ------------- | ----------------------------------------------------------------------------------------------- | -------------------------------------------------------------- | | SETR-Naive | 512x512 | 16 | 160k | Val | 48.06 | 48.80 | google drive | config | | SETR-MLA | 512x512 | 8 | 160k | val | 47.79 | 50.03 | google drive | config | | SETR-MLA | 512x512 | 16 | 160k | val | 48.64 | 50.28 | google drive | config | | SETR-MLA-Deit | 512x512 | 16 | 160k | val | 46.15 | 47.71 | google drive | config | | SETR-PUP | 512x512 | 16 | 160k | val | 48.62 | 50.09 | google drive | config | | SETR-PUP-Deit | 512x512 | 16 | 160k | val | 46.34 | 47.30 | google drive | config |

Pascal Context

| Method | Crop Size | Batch size | iteration | set | mIoU | mIoU(ms+flip) | model | Config | | ------------- | --------- | ---------- | --------- | --- | ----- | ------------- | ----------------------------------------------------------------------------------------------- | --------------------------------------------------------------------- | | SETR-Naive | 480x480 | 16 | 80k | val | 52.89 | 53.61 | google drive | config | | SETR-MLA | 480x480 | 8 | 80k | val | 54.39 | 55.39 | google drive | config | | SETR-MLA | 480x480 | 16 | 80k | val | 55.01 | 55.83 | google drive | config | | SETR-MLA-DeiT | 480x480 | 16 | 80k | val | 52.91 | 53.74 | google drive | config | | SETR-PUP | 480x480 | 16 | 80k | val | 54.37 | 55.27 | google drive | config | | SETR-PUP-DeiT | 480x480 | 16 | 80k | val | 52.00 | 52.50 | google drive | config |

HLG

ImageNet-1K

HLG classification is under folder hlg-classification/.

| Model | Resolution | Params | FLOPs | Top-1 % | Config | Pretrained Model | | ---------- | :--------: | :----: | :---: | :-----: | ---------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- | | HLG-Tiny | 224 | 11M | 2.1G | 81.1 | hlg_tiny_224.yaml | google drive | | HLG-Small | 224 | 24M | 4.7G | 82.3 | hlg_small_224.yaml | google drive | | HLG-Medium | 224 | 43M | 9.0G | 83.6 | hlg_medium_224.yaml | [google drive](https://drive.google.com/file/d/1nax3Ez3