Weakly Supervised Semantic Segmentation list

This repository contains lists of state-or-art weakly supervised semantic segmentation works. Papers and resources are listed below according to supervision types.

There are some personal views and notes, just ignore if not interested.

Last update 2019/4

• [x] Paper list
  • [x] instance
  • [x] box
  • [x] one-shot
  • [x] others
• [x] Resources

some unsupervised segment proposal methods and datasets here.

CVPR 2018 Tutorial : WSL web&ppt, Part1 ,Part2

Typical weak supervised segmentation problems

| No | Supervision | Difficulty | Domain | Core issues | | -- | ----------- | ---------- | ------ | ----------- | | 1 | Bounding box | middle | annotated classes | transfer learning | | 2 | One-shot segment | middle | similar objects | one-shot learning | | 3 | Image/video label | hard | annotated classes | transfer learning | | 4 | Others | n/a | n/a | n/a |

<h2 id="1">1.Bounding box supervision</h2>

• Box-driven Class-wise Region Masking and Filling Rate Guided Loss for Weakly Supervised Semantic Segmentation, CVPR 2019

• Learning to Segment Every Thing, CVPR 2018

  :Learning weight transfer from well-annotated subset, transfer class-specific weights(output layers) from detection and classification branch, based on Mask-RCNN

• Pseudo Mask Augmented Object Detection, CVPR 2018

  :State-of-art weakly supervised instance segmentation with bounding box annotation. EM optimizes pseudo mask and segmentation parameter like Boxsup. Graphcut on superpixel is employed to refine pseudo mask.

• Simple Does It: Weakly Supervised Instance and Semantic Segmentation, CVPR 2017 [web] [ref-code][supp]

  :Grabcut+(HED bounday) and MCG , train foreground segmentation network directly with generated mask semantic segmentaion, sensitive to env(quality) of training images.

• Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation, ICCV 2015

  :Based on CRF refine, EM seems not work

• BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation, ICCV 2015

  :Iteratively update parameters and region proposal labels, proposals are selected by network output masks

• Deepcut: Object segmentation from bounding box annotations using convolutional neural networks, TMI 2017

• Adversarial Learning for Semi-Supervised Semantic Segmentation, BMVC 2018, [code]

<h2 id="2">2.One-Shot segmentation supervision</h2>

DAVIS Challenge: http://davischallenge.org/

: Davis17/18(Semi-supervised Video segmentation task), Davis16 is video salient object segmentation without the first frame annotations.

• Fast and Accurate Online Video Object Segmentation via Tracking Parts, CVPR 2018(Spotlight) [code]

  :state-of-art, 82.4%/1.8s 77.9%/0.6s

• OSVOS: One-Shot Video Object Segmentation, CVPR 2017 [web][code]

  :milestone, fine-tuning parent network with the first frame mask, 79.8%/10s

<h2 id="3">3.Image/video label supervision</h2>

• Self-produced Guidance for Weakly-supervised Object Localization, ECCV 2018

• Convolutional Simplex Projection Network (CSPN) for Weakly Supervised Semantic Segmentation, BMVC 2018

• Weakly Supervised Instance Segmentation using Class Peak Response, CVPR 2018(Spotlight)

  :state-of-art practice for instance seg with only class label.

<img src="img/prm.png" alt="prm" title="prm" width="750" height="300" />

• Weakly-Supervised Semantic Segmentation by Iteratively Mining Common Object Features, CVPR 2018

  :Superpixel-> RegionNet(RoI classfier)-> Saliency refine, iteratively update with PixelNet(FCN)

• Revisiting Dilated Convolution: A Simple Approach for Weakly- and SemiSupervised Semantic Segmentation, CVPR 2018(Spotlight)

• Weakly-Supervised Semantic Segmentation Network With Deep Seeded Region Growing, CVPR 2018 [web][code]

• Adversarial Complementary Learning for Weakly Supervised Object Localization, CVPR 2018

• Learning Pixel-level Semantic Affinity with Image-level Supervision for Weakly Supervised Semantic Segmentation, CVPR 2018

• Multi-Evidence Filtering and Fusion for Multi-Label Classification, Object Detection and Semantic Segmentation Based on Weakly Supervised Learning, CVPR 2018

• Weakly Supervised Semantic Segmentation using Web-Crawled Videos, CVPR 2017(Spotlight) [web]

• Object Region Mining with Adversarial Erasing: A Simple Classification to Semantic Segmentation Approach, CVPR 2017

• WILDCAT: Weakly Supervised Learning of Deep ConvNets for Image Classification, Pointwise Localization and Segmentation, CVPR 2017 [web][code]

• Learning random-walk label propagation for weakly-supervised semantic segmentation, CVPR 2017(Oral)

• Combining Bottom-Up, Top-Down, and Smoothness Cues for Weakly Supervised Image Segmentation, CVPR 2017

• Weakly Supervised Semantic Segmentation Using Superpixel Pooling Network, AAAI 2017

• Learning from Weak and Noisy Labels for Semantic Segmentation, PAMI 2017

• Learning to Segment Human by Watching YouTube, PAMI 2017

• Seed, Expand, Constrain: Three Principles for Weakly-Supervised Image Segmentation, ECCV 2016 [code]

• Backtracking ScSPM Image Classifier for Weakly Supervised Top-down Saliency, CVPR 2016, TIP 2018 Version

• Constrained Convolutional Neural Networks for Weakly Supervised Segmentation, ICCV 2015 [code]

• From Image-level to Pixel-level Labeling with Convolutional Networks, CVPR 2015

Resource

• Yunchao Wei talk in Chinese about WSL with image label

Arxiv paper

• Learning to Exploit the Prior Network Knowledge for Weakly-Supervised Semantic Segmentation, Arxiv1804

• Weakly Supervised Object Discovery by Generative Adversarial & Ranking Networks, Arxiv 1711

<h2 id="3.1">3.1 Deep activation</h2>

| Propagate method | Papers | | ---------------- | ------ | | Global Max Pooling(GMP) | Is object localization for free? - Weakly-supervised learning with convolutional neural networks,CVPR 2015 | | Global Average Pooling(GAP) | Learning Deep Features for Discriminative Localization CVPR 2016| | Log-sum-exponential Pooling(LSE)| ProNet: Learning to Propose Object-specific Boxes for Cascaded Neural Networks,CVPR 2016| | Global Weighted Rank Pooling(GWRP) | SEC ECCV 2016| | Global rank Max-Min Pooling(GRP)| WILDCAT, CVPR 2017|

<h2 id="3.1">3.2 Weakly supervised Detection / Localization(TODO) </h2>

• PCL: Proposal Cluster Learning for Weakly Supervised Object Detection, PAMI 2018 [code]

• Weakly Supervised Region Proposal Network and Object Detection, ECCV 2018

• TS2C: Tight Box Mining with Surrounding Segmentation Context for Weakly Supervised Object Detection, ECCV 2018

• Zigzag Learning for Weakly Supervised Object Detection, CVPR 2018

• [W2F: A Weakly-Supervised to Fully-Supervised Framework for Object Detection](https://ivul.kaust.edu.sa/Documents/Publications/2018/W2F%20A%20Weakly-Supervised%20to%20Fully-Supervised%20Fr