Awesome-Open-Vocabulary-Semantic-Segmentation

If you find this project helpful, please consider giving it a star ⭐.

Contents

<!-- vim-markdown-toc GitLab -->

• Open-Vocabulary Semantic Segmentation (mainly updated by @tbh3223)

  • Fully-supervised Methods
  • Weakly-supervised Methods
  • Training-free Methods
  • Others

• Zero-shot Semantic Segmentation (mainly updated by @tbh3223)

• Referring-Image-Segmentation (mainly updated by @ghost-000)

  • Fully-Supervised Methods
  • Weakly-Supervised Methods

• Open-Vocabulary Object Detection (mainly updated by @tbh3223)

• Universal Segmentation and Related Work (mainly updated by @tbh3223)

• Other Open-Vocabulary Related Work

• Related Survey

<!-- vim-markdown-toc -->

Open-Vocabulary Semantic Segmentation

Fully-Supervised Open-Vocabulary Semantic Segmentation

The model is trained on fully-supervised semantic segmentation datasets with pixel-level annotations (e.g., COCO Stuff dataset).

1. <span id = "1001">[LSeg]</span> | ICLR'22 | Language-driven Semantic Segmentation | [pdf] | [code]
2. <span id = "1002">[OpenSeg]</span> | ECCV'22 | Scaling Open-vocabulary Image Segmentation with Image-level Labels | [pdf] | [code]
3. <span id = "1003">[Xu et al.]</span> | ECCV'22 | A Simple Baseline for Open-Vocabulary Semantic Segmentation with Pre-trained Vision-language Model | [pdf] | [code]
4. <span id = "1003">[SegCLIP]</span> | ICML'23 | SegCLIP: Patch Aggregation with Learnable Centers for Open-Vocabulary Semantic Segmentation | [pdf] | [code]
5. <span id = "1003">[MaskCLIP]</span> | ICML'23 | Open-Vocabulary Universal Image Segmentation with MaskCLIP | [pdf] | [code]
6. <span id = "1003">[OVSeg]</span> | CVPR'23 | Open-Vocabulary Semantic Segmentation with Mask-adapted CLIP | [pdf] | [code]
7. <span id = "1003">[X-Decoder]</span> | CVPR'23 | Generalized Decoding for Pixel, Image, and Language | [pdf] | [code]
8. <span id = "1003">[SAN]</span> | CVPR'23(Highlight) | Side Adapter Network for Open-Vocabulary Semantic Segmentation | [pdf] | [code]
9. <span id = "1003">[SAN]</span> | TAPMI'23 | SAN: Side Adapter Network for Open-vocabulary Semantic Segmentation | [pdf] | [code]
10. <span id = "1003">[ODISE]</span> | CVPR'23 | Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Models | [pdf] | [code]
11. <span id = "1003">[FreeSeg]</span> | CVPR'23 | FreeSeg: Unified, Universal and Open-Vocabulary Image Segmentation | [pdf] | [code]
12. <span id = "1003">[OpenSeeD]</span> | ICCV'23 | A Simple Framework for Open-Vocabulary Segmentation and Detection | [pdf] | [code]
13. <span id = "1003">[GKC]</span> | ICCV'23 | Global Knowledge Calibration for Fast Open-Vocabulary Segmentation | [pdf]
14. <span id = "1003">[OPSNet]</span> | ICCV'23 | Open-vocabulary Panoptic Segmentation with Embedding Modulation | [pdf] | [code]
15. <span id = "2005">[MasQCLIP]</span> | ICCV'23 | MasQCLIP for Open-Vocabulary Universal Image Segmentation | [pdf]
16. <span id = "2005">[DeOP]</span> | ICCV'23 | Open Vocabulary Semantic Segmentation with Decoupled One-Pass Network | [pdf] | [code]
17. <span id = "2005">[Li et al.]</span> | ICCV'23 | Open-vocabulary Object Segmentation with Diffusion Models | [pdf] | [code]
18. <span id = "1003">[HIPIE]</span> | NeurIPS'23 | Hierarchical Open-vocabulary Universal Image Segmentation | [pdf] | [code]
19. <span id = "1003">[FC-CLIP]</span> | NeurIPS'23 | Convolutions Die Hard: Open-Vocabulary Segmentation with Single Frozen Convolutional CLIP | [pdf] | [code]
20. <span id = "1003">[MAFT]</span> | NeurIPS'23 | Learning Mask-aware CLIP Representations for Zero-Shot Segmentation | [pdf] | [code]
21. <span id = "1003">[ADA]</span> | NeurIPS'23 | Open-Vocabulary Semantic Segmentation via Attribute Decomposition-Aggregation | [pdf]
22. <span id = "1003">[Dao et al.]</span> | TMM | Class Enhancement Losses with Pseudo Labels for Open-Vocabulary Semantic Segmentation | [pdf]
23. <span id = "1003">[SELF-SEG]</span> | ArXiv'23.12 | Self-Guided Open-Vocabulary Semantic Segmentation | [pdf]
24. <span id = "1003">[OpenSD]</span> | ArXiv'23.12 | OpenSD: Unified Open-Vocabulary Segmentation and Detection | [pdf] | [code]
25. <span id = "1003">[SILC]</span> | ArXiv'23.12 | SILC: Improving Vision Language Pretraining with Self-Distillation | [pdf]
26. <span id = "1003">[CLIPSelf]</span> | ICLR'24(Spotlight) | CLIPSelf: Vision Transformer Distills Itself for Open-Vocabulary Dense Prediction | [pdf] | [code]
27. <span id = "1003">[RENOVATE]</span> | ArXiv'24.03 | Renovating Names in Open-Vocabulary Segmentation Benchmarks | [pdf]
28. <span id = "1003">[DreamCLIP]</span> | ECCV'24 | DreamLIP: Language-Image Pre-training with Long Captions | [pdf] | [code]
29. <span id = "1003">[CAT-Seg]</span> | CVPR'24 | CAT-Seg : Cost Aggregation for Open-Vocabulary Semantic Segmentation | [pdf] | [code]
30. <span id = "1003">[SED]</span> | CVPR'24 | SED: A Simple Encoder-Decoder for Open-Vocabulary Semantic Segmentation | [pdf] | [code]
31. <span id = "1003">[SCAN]</span> | CVPR'24 | Open-Vocabulary Segmentation with Semantic-Assisted Calibration | [pdf] | [code]
32. <span id = "1003">[OpenTrans]</span> | CVPR'24 | Transferable and Principled Efficiency for Open-Vocabulary Segmentation | [pdf] | [code])
33. <span id = "1003">[H-CLIP]</span> | ArXiv'24.05 | Parameter-efficient Fine-tuning in Hyperspherical Space for Open-vocabulary Semantic Segmentation | [pdf]
34. <span id = "1003">[OpenDAS]</span> | ArXiv'24.05 | OpenDAS: Domain Adaptation for Open-Vocabulary Segmentation | [pdf]
35. <span id = "1003">[USE]</span> | CVPR'24 | USE: Universal Segment Embeddings for Open-Vocabulary Image Segmentation | [pdf]
36. <span id = "1003">[EBSeg]</span> | CVPR'24 | Open-Vocabulary Semantic Segmentation with Image Embedding Balancing | [pdf] | [code])
37. <span id = "1003">[MAFT+]</span> | ECCV'24 | Collaborative Vision-Text Representation Optimizing for Open-Vocabulary Segmentation | [pdf] | [code])
38. <span id = "1003">[R-Adapter]</span> | **EC