Awesome Uncertainty in Deep learning

<div align="center">

</div>

This repo is a collection of awesome papers, codes, books, and blogs about Uncertainty and Deep learning.

:star: Feel free to star and fork. :star:

If you think we missed a paper, please open a pull request or send a message on the corresponding GitHub discussion. Tell us where the article was published and when, and send us GitHub and ArXiv links if they are available.

We are also open to any ideas for improvements!

<h2> Table of Contents </h2>

• Awesome Uncertainty in Deep learning
• Papers
  • Surveys
  • Theory
  • Bayesian-Methods
  • Ensemble-Methods
  • Sampling/Dropout-based-Methods
  • Post-hoc-Methods/Auxiliary-Networks
  • Data-augmentation/Generation-based-methods
  • Output-Space-Modeling/Evidential-deep-learning
  • Deterministic-Uncertainty-Methods
  • Quantile-Regression/Predicted-Intervals
  • Conformal Predictions
  • Calibration/Evaluation-Metrics
  • Misclassification Detection & Selective Classification
  • Anomaly-detection and Out-of-Distribution-Detection
  • Uncertainty sources & Aleatoric and Epistemic Uncertainty Disentenglement
  • Uncertainty Quantification in Multimodal Models / GenAI
  • Applications
    • Classification and Semantic-Segmentation
    • Regression
    • Object detection
    • Domain adaptation
    • Semi-supervised and Active Learning
    • Natural Language Processing
    • Others
• Datasets and Benchmarks
• Libraries
  • Python
  • PyTorch
  • JAX
  • TensorFlow
• Lectures and tutorials
• Books
• Other Resources

Papers

Surveys

Conference

• Benchmarking Uncertainty Disentanglement: Specialized Uncertainties for Specialized Tasks [NeurIPS2024 - [PyTorch]
• A Comparison of Uncertainty Estimation Approaches in Deep Learning Components for Autonomous Vehicle Applications [AISafety Workshop 2020]

Journal

• A survey of uncertainty in deep neural networks [Artificial Intelligence Review 2023] - [GitHub]
• Prior and Posterior Networks: A Survey on Evidential Deep Learning Methods For Uncertainty Estimation [TMLR2023]
• A Survey on Uncertainty Estimation in Deep Learning Classification Systems from a Bayesian Perspective [ACM2021]
• Ensemble deep learning: A review [Engineering Applications of AI 2021]
• A review of uncertainty quantification in deep learning: Techniques, applications and challenges [Information Fusion 2021]
• Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods [Machine Learning 2021]
• Predictive inference with the jackknife+ [The Annals of Statistics 2021]
• Uncertainty in big data analytics: survey, opportunities, and challenges [Journal of Big Data 2019]

Arxiv

• A System-Level View on Out-of-Distribution Data in Robotics [arXiv2022]
• A Survey on Uncertainty Reasoning and Quantification for Decision Making: Belief Theory Meets Deep Learning [arXiv2022]

Theory

Conference

• Exploring and Exploiting Model Uncertainty in Bayesian Optimization [NeurIPS2025]
• A Rigorous Link between Deep Ensembles and (Variational) Bayesian Methods [NeurIPS2023]
• Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning [ICLR2023]
• Unmasking the Lottery Ticket Hypothesis: What's Encoded in a Winning Ticket's Mask? [ICLR2023]
• Probabilistic Contrastive Learning Recovers the Correct Aleatoric Uncertainty of Ambiguous Inputs [ICML2023] - [PyTorch]
• On Second-Order Scoring Rules for Epistemic Uncertainty Quantification [ICML2023]
• Neural Variational Gradient Descent [AABI2022]
• Top-label and multiclass-to-binary reductions [ICLR2022]
• Bayesian Model Selection, the Marginal Likelihood, and Generalization [ICML2022]
• With malice towards none: Assessing uncertainty via equalized coverage [AIES 2021]
• Uncertainty in Gradient Boosting via Ensembles [ICLR2021] - [PyTorch]
• Repulsive Deep Ensembles are Bayesian [NeurIPS2021] - [PyTorch]
• Bayesian Optimization with High-Dimensional Outputs [NeurIPS2021]
• Residual Pathway Priors for Soft Equivariance Constraints [NeurIPS2021]
• Dangers of Bayesian Model Averaging under Covariate Shift [NeurIPS2021] - [TensorFlow]
• A Mathematical Analysis of Learning Loss for Active Learning in Regression [CVPR Workshop2021]
• Why Are Bootstrapped Deep Ensembles Not Better? [NeurIPS Workshop]
• Deep Convolutional Networks as shallow Gaussian Processes [ICLR2019]
• On the accuracy of influence functions for measuring group effects [NeurIPS2018]
• To Trust Or Not To Trust A Classifier [NeurIPS2018] - [Python]
• Understanding Measures of Uncertainty for Adversarial Example Detection [UAI2018]

Journal

• Martingale posterior distributions [Royal Statistical Society Series B]
• A Unified Theory of Diversity in Ensemble Learning [JMLR2023]
• Multivariate Uncertainty in Deep Learning [TNNLS2021]
• A General Framework for Uncertainty Estimation in Deep Learning [RAL2020]
• Adaptive nonparametric confidence sets [Ann. Statist. 2006]

Arxiv

• Ensembles for Uncertainty Estimation: Benefits of Prior Functions and Bootstrapping [arXiv2022]
• Efficient Gaussian Neural Processes for Regression [arXiv2021]
• Dense Uncertainty Estimation [arXiv2021] - [PyTorch]
• A higher-order swiss army infinitesimal jackknife [arXiv2019]

Bayesian-Methods

Conference

• Quantifying Uncertainty in the Presence of Distribution Shifts [NeurIPS2025]
• Training Bayesian Neural Networks with Sparse Subspace Variational Inference [ICLR2024]
• Variational Bayesian Last Layers [ICLR2024]
• A Symmetry-Aware Exploration of Bayesian Neural Network Posteriors [ICLR2024]
• Beyond Unimodal: Generalising Neural Processes for Multimodal Uncertainty Estimation [NeurIPS2023]
• Uncertainty-aware Unsupervised Video Hashing [AISTATS2023] - [PyTorch]
• Gradient-ba