MI-AOD

Language: 简体中文 | English

<!-- TOC -->

• Introduction
• Getting Started
• Model Zoo
• Repository Contributor
• License
• Citation

<!-- TOC -->

Introduction

This is the code for Multiple Instance Active Learning for Object Detection, CVPR 2021.

Task Description

In this paper, we propose Multiple Instance Active Object Detection (MI-AOD), to select the most informative images for detector training by observing instance-level uncertainty.

The process of active object detection (active learning for object detection) is shown in the figure below.

First, a small set of images (the labeled set) with instance labels and a large set of images (the unlabeled set) without labels are given. For each image, the label consists of bounding boxes and categories for objects of interest.

A detection model is firstly initialized by using the labeled set {}. With the initialized model , active learning targets at selecting a set of images from to be manually labeled and merging them with for a new labeled set , i.e., . The selected image set should be the most informative, i.e., can improve the detection performance as much as possible.

The informativeness in the figure above is embodied as the uncertainty. That is to say, with the sample in input into the current model, if the output score of the model for each class is more uniform, the uncertainty of this sample is higher.

Based on the updated labeled set , the task model is retrained and updated to . The model training and sample selection repeat some cycles until the size of labeled set reaches the annotation budget.

Illustration

MI-AOD defines an instance uncertainty learning module, which leverages the discrepancy of two adversarial instance classifiers trained on the labeled set to predict instance uncertainty of the unlabeled set. MI-AOD treats unlabeled images as instance bags and feature anchors in images as instances, and estimates the image uncertainty by re-weighting instances in a multiple instance learning (MIL) fashion. Iterative instance uncertainty learning and re-weighting facilitate suppressing noisy instances, toward bridging the gap between instance uncertainty and image-level uncertainty.

Here and here are more paper interpretation in Chinese.

Innovation

• Creatively design a tailor-made method for the active learning + object detection task for the first time.

• Achieve 93.5% of the performance of 100% labeled data by using only 20% of the labeled data on the PASCAL VOC dataset.

• Active learning was applied to the MS COCO dataset for the first time with the best performance.

• The idea is clear and simple, and can be generalized to any types of detection models.

Boarder Impact

MI-AOD focuses on object detection (OD), but it can be also generalized to:

• any other visual object detection tasks (tiny object detection, pedestrian detection, medical image detection),
• any other computer vision tasks (human pose detection, semantic/instance segmentation, temporal action detection),
• and any other machine learning tasks like natural language processing,

by combining active learning with these tasks. This bottom-up and top-down idea can be generalized and applied to any of these tasks.

Notice that active learning works a lot for visual object detection in MI-AOD, other learning methods with less supervision can be combined with it, such as:

• active few-shot learning,
• active semi-/weak/self supervised learning,
• active transfer learning,
• active reinforcement learning,
• active incremental learning,

and so on. These combination of active learning and other learning method can promote each other to a greater extent.

Getting Started

Installation

Please refer to Installation.md for installation.

Data Preparation

Please download VOC2007 datasets ( trainval + test ) and VOC2012 datasets ( trainval ) from:

VOC2007 ( trainval ): http://host.robots.ox.ac.uk/pascal/VOC/voc2007/VOCtrainval_06-Nov-2007.tar

VOC2007 ( test ): http://host.robots.ox.ac.uk/pascal/VOC/voc2007/VOCtest_06-Nov-2007.tar

VOC2012 ( trainval ): http://host.robots.ox.ac.uk/pascal/VOC/voc2012/VOCtrainval_11-May-2012.tar

And after that, please ensure the file directory tree is as below:

├── VOCdevkit
│   ├── VOC2007
│   │   ├── Annotations
│   │   ├── ImageSets
│   │   ├── JPEGImages
│   ├── VOC2012
│   │   ├── Annotations
│   │   ├── ImageSets
│   │   ├── JPEGImages

You may also use the following commands directly:

cd $YOUR_DATASET_PATH
wget http://host.robots.ox.ac.uk/pascal/VOC/voc2007/VOCtrainval_06-Nov-2007.tar
wget http://host.robots.ox.ac.uk/pascal/VOC/voc2007/VOCtest_06-Nov-2007.tar
wget http://host.robots.ox.ac.uk/pascal/VOC/voc2012/VOCtrainval_11-May-2012.tar
tar -xf VOCtrainval_06-Nov-2007.tar
tar -xf VOCtest_06-Nov-2007.tar
tar -xf VOCtrainval_11-May-2012.tar

If you want to use SSD detectors instead of RetinaNet detectors, you can replace the configuration file in this repository as below:

mv configs/MIAOD.py configs/MIAOD_Retina.py
mv configs/MIAOD_SSD.py configs/MIAOD.py

For the SSD detector, because the vgg16 pre-trained model link provided in the mmcv 1.0.5 package is no longer available, the json file that loads the pre-trained model link needs to be updated to the latest version:

wget https://github.com/open-mmlab/mmcv/raw/master/mmcv/model_zoo/open_mmlab.json
cp -v open_mmlab.json $YOUR_ANACONDA_PATH/envs/miaod/lib/python3.7/site-packages/mmcv/model_zoo/

Please change the $YOUR_ANACONDA_PATH to your actual Anaconda3 installation directory. Usually it would be ~/anaconda3.

After that, please modify the corresponding dataset directory, they are located in:

Line 2 of configs/MIAOD.py: data_root='$YOUR_DATASET_PATH/VOCdevkit/'
Line 2 of configs/_base_/voc0712.py: data_root='$YOUR_DATASET_PATH/VOCdevkit/'

Please change the $YOUR_DATASET_PATHs above to your actual dataset directory (i.e., the directory where you intend to put the downloaded VOC tar file).

And please use the absolute path (i.e., start with /) but not a relative path (i.e., start with ./ or ../）.

Please refer to here for the information of other variables and parameters.

Please refer to here for the data preparation on MS COCO.

Train and Test

We recommend you to use a GPU but not a CPU to train and test, because it will greatly shorten the time.

And we also recommend you to use a single GPU, because the usage of multi-GPU may result in erro