Generalized Out-of-Distribution Detection: A Survey

   

1. Overview

This repository is with our survey paper:

Title: Generalized Out-of-Distribution Detection: A Survey <br> Authors: Jingkang Yang¹, Kaiyang Zhou¹, Yixuan Li², Ziwei Liu¹ <br> Institutions: ¹MMLab@NTU, ²University of Wisconsin-Madison.

This survey comprehensively reviews the similar topics of outlier detection (OD), anomaly detection (AD), novelty detection (ND), open set recognition (OSR), and out-of-distribution (OOD) detection, extensively compares their commomality and differences, and eventually unifies them under a big umbrella of "generalized OOD detection" framework.

We hope that this survey can help readers and participants better understand the open-world field centered on OOD detection. At the same time, it urges future work to learn, compare, and develop ideas and methods from the broader scope of generalized OOD detection, with clear problem definition and proper benchmarking.

We prepare this repository for the following two reasons:

1. We consider it an awesome list to easily access the references mentioned in the paper Table 1. We also believe this list will continue to include more promising works as new works appear. Please feel free to nominate good related works with Pull Requests.
2. We hope this repository becomes a discussion panel for readers to ask questions, raise concerns, and make constructive comments for the broad generalized OOD detection field. Please feel free to post your ideas in the Issues.

We are also planning to build an evaluation benchmark to compare representative generalized OOD detection methods from every sub-task to further unify the field. The work will be collaborated with SenseTime EIG Research, which recently have many full-time researcher openings for this benchmarking project and other OOD-related research. Check their Recruitment Info for more information.

| :-----------------------------:|:-------------------------: Fig.1.1: Two kinds of distribution shift to assist better understanding of our framework. | Fig.1.2: Taxonomy diagram of generalized OOD detection framework.

2. Taxonomy

• 2.1 Anomaly Detection
  • 2.1.1 Sensory Anomaly Detection
  • 2.1.2 Semantic Anomaly Detection
• 2.2 Novelty Detection
  • 2.2.1 One-Class Novelty Detection
  • 2.2.2 Multi-Class Novelty Detection
• 2.3 Open Set Recognition
• 2.4 Out-of-Distribution Detection
• 2.5 Outlier Detection
• 2.6 Discussion

3. Anomaly Detection & One-Class Novelty Detection

• 3.1 Density-based Methods
  • 3.1.1 Classic Density Estimation
  • 3.1.2 NN-based Density Estimation
  • 3.1.3 Energy-based Model
  • 3.1.4 Frequency-based Model
• 3.2 Reconstruction-based Methods
  • 3.2.1 Sparse Representation Methods
  • 3.2.2 Reconstruction-Error Methods
• 3.3 Classification-based Methods
  • 3.3.1 One-Class Classification
  • 3.3.2 Positive-Unlabeled Learning
  • 3.3.3 Self-Supervised Learning
• 3.4 Distance-based Methods
• 3.5 Gradient-based Methods
• 3.6 Discussion and Theoretical Analysis

4. Multi-Class Novelty Detection & Open Set Recognition

• 4.1 Classfication-based Methods
  • 4.1.1 EVT-based Calibration
  • 4.1.2 EVT-free Calibration
  • 4.1.3 Unknown Class Generation
  • 4.1.4 Label Space Redesign
• 4.2 Distance-based Methods
• 4.3 Reconstruction-based Methods
  • 4.3.1 Sparse Representation Methods
  • 4.3.2 Reconstruction-Error Methods

5. Out-of-Distribution Detection

• 5.1 Classfication-based Methods
  • 5.1.0 Baseline
  • 5.1.1 Output-based Methods
    • 5.1.1.1 Post-hoc Detection
    • 5.1.1.2 Confidence Enhancement
    • 5.1.1.3 Outlier Exposure
  • 5.1.2 OOD Data Generation
  • 5.1.3 Gradient-based Methods
  • 5.1.4 Bayesian Models
  • 5.1.5 Large-scale OOD Detection
• 5.2 Density-based Methods
• 5.3 Distance-based Methods

6. Outlier Detection

• 6.1 Density-based Methods
• 6.2 Distance-based Methods
  • 6.2.1 Cluster-based Methods
  • 6.2.2 Graph-based Methods
• 6.3 Classification-based Methods

7. Challenges and Future Direction

• 7.1 Challenges
  • 7.1.1 Proper Evaluation and Benchmarking
  • 7.1.2 Outlier-free OOD Detection
  • 7.1.3 Tradeoff Between Classification and OOD Detection
  • 7.1.4 Real-world Benchmarks and Evaluations
• 7.2 Future Directions
  • [7.2.1 Methodologies across Sub-tasks](https://github.com/Jingkang50/OOD_Detection_Survey/blob/main/7