Conformal Prediction: The Definitive Resource

⭐ The definitive resource for conformal prediction: methods, libraries, tutorials, benchmarks, and production guides.

🔥 Updated regularly with the latest conformal prediction research (2025–2026). The canonical repository for conformal prediction, maintained by a leading researcher in the field — used by researchers, practitioners, and production teams worldwide.

Topics covered: conformal prediction · conformal inference · prediction intervals · uncertainty quantification · model calibration · distribution-free inference · selective prediction · trustworthy machine learning · LLM uncertainty · time series forecasting · anomaly detection

Conformal prediction (also called conformal inference) is a distribution-free, model-agnostic framework for quantifying predictive uncertainty. Instead of returning a single point prediction, a conformal predictor returns a prediction set (for classification) or a prediction interval (for regression) that is guaranteed to contain the true label with a user-chosen probability — for example 90% or 95% — under the mild assumption that the data are exchangeable. These guarantees hold in finite samples, for any underlying model (random forests, gradient boosting, deep neural networks, large language models), and without any distributional assumptions.

🧭 New here? Follow this path

| Step | What to do | Where to look | |---:|---|---| | 1 | Understand the core idea in 10 minutes | Tutorials | | 2 | Watch a 30-minute talk | Videos and Talks | | 3 | Read a gentle introduction | Articles | | 4 | Try it in code | Libraries in Python · Quickstart | | 5 | Go deep | Courses · Books · Papers |

---

📘 Learn from the Experts — Book & Course by Dr. Valeriy Manokhin

<table> <tr> <td width="50%" valign="top">

🎓 Applied Conformal Prediction — Live Course on Maven

A hands-on, cohort-based course taught by Dr. Valeriy Manokhin (PhD, Machine Learning; author of Practical Guide to Applied Conformal Prediction; PhD student of Prof. Vladimir Vovk — the creator of conformal prediction).

You will learn how to apply conformal prediction to regression, classification, time series, anomaly detection, and LLM uncertainty, with production-ready Python code and real datasets.

🔥 Enroll in the next cohort → 📩 Register interest for upcoming cohorts →

</td> <td width="50%" valign="top">

📚 Applied Conformal Prediction — The Book

The definitive practitioner's book on conformal prediction — from first principles to industry-grade applications, with code you can run today. Written by Dr. Valeriy Manokhin.

🚀 Applied Conformal Prediction — Pro Edition ⭐ Recommended The most comprehensive edition: extended chapters, advanced methods, and production-grade code.

📘 Applied Conformal Prediction — Standard Edition The core book covering regression, classification, time series, anomaly detection, and LLM uncertainty.

Both editions are available exclusively on Gumroad.

🌐 valeriy.ai — author's website with research, papers, and updates.

</td> </tr> </table>

"The most comprehensive practical resource on conformal prediction currently available."

---

What is Conformal Prediction?

Conformal prediction wraps any predictive model and converts its outputs into statistically valid prediction sets or intervals. The recipe is simple:

1. Train any model on a training set.
2. Score a held-out calibration set with a nonconformity function (e.g. absolute residual for regression, 1 − softmax for classification).
3. At test time, output every label whose nonconformity score is no worse than the (1 − α) quantile of the calibration scores.

The resulting prediction set is guaranteed to cover the true label with probability at least 1 − α, regardless of the model or the data distribution. This makes conformal prediction one of the most practical tools for uncertainty quantification, trustworthy machine learning, selective prediction, and safe deployment of AI systems.

Quickstart

A minimal end-to-end example using crepes:

import numpy as np
from crepes import ConformalRegressor
from sklearn.ensemble import RandomForestRegressor
from sklearn.datasets import fetch_california_housing
from sklearn.model_selection import train_test_split

X, y = fetch_california_housing(return_X_y=True)
X_train, X_rest, y_train, y_rest = train_test_split(X, y, test_size=0.4, random_state=0)
X_cal,  X_test, y_cal,  y_test  = train_test_split(X_rest, y_rest, test_size=0.5, random_state=0)

model = RandomForestRegressor(n_estimators=200, random_state=0).fit(X_train, y_train)
residuals_cal = np.abs(y_cal - model.predict(X_cal))

cr = ConformalRegressor().fit(residuals=residuals_cal)
intervals = cr.predict(y_hat=model.predict(X_test), confidence=0.9)  # 90% intervals

Runnable notebooks live in examples/. See CONTRIBUTING.md to add resources.

---

About this list

This collection is curated by Dr. Valeriy Manokhin, who completed his PhD in Machine Learning under the supervision of Prof. Vladimir Vovk — the creator of conformal prediction. It is the culmination of resources gathered since 2015. Conformal prediction has gone from a niche framework to a mainstream method for uncertainty quantification, with dedicated tracks at ICML 2021 and ICML 2022, a NeurIPS 2022 keynote 'Conformal Prediction in 2022' by Prof. Emmanuel Candes, and the long-running COPA conference.

"Conformal Prediction ideas are THE answer to UQ, I think it's the best I have seen — its simple, generalisable etc." — Prof. Michael I. Jordan, ICML 2021

"The beauty of the conformal thing is how simple it is to do it and how general it is." — Prof. Larry Wasserman, Carnegie Mellon

"Conformal inference methods are becoming all the rage in academia and industry alike." — Prof. Emmanuel Candes, Stanford

The author's course 'Applied Conformal Prediction' is open for enrollment on Maven — enroll in the next cohort or register interest. The companion book 'Applied Conformal Prediction: Reliable Uncertainty Quantification for Real-World Machine Learning' is available on Gumroad.

<a href="https://www.buymeacoffee.com/valeman" target="_blank"><img src="https://cdn.buymeacoffee.com/buttons/default-orange.png" alt="Buy Me A Coffee" height="41" width="174"></a>

Citation

@software{manokhin_valery_2022_6467204,
  author  = {Manokhin, Valery},
  title   = {Awesome Conformal Prediction},
  month   = apr,
  year    = 2022,
  publisher = {Zenodo},
  version = {v1.0.0},
  doi     = {10.5281/zenodo.6467204},
  url     = {https://doi.org/10.5281/zenodo.6467204}
}

Licensed CC BY-NC-ND 4.0. Academic work must cite.

---

💬 Open Questions in Conformal Prediction

These are active research and practitioner questions — contributions, discussion, and PRs welcome:

• What is the best conformal prediction library today for production use?
• Are transformers and LLMs fully compatible with conformal prediction guarantees?
• How should practitioners handle distribution shift and covariate drift in practice?
• What are the tightest known prediction intervals for deep learning models?
• How do conformal methods compare with Bayesian uncertainty quantification on modern benchmarks?

👉 Open a discussion or issue to contribute.

Table of Contents

🚀 Start Here — Learn Conformal Prediction

• Tutorials
• Courses
• Books
• Articles
• Videos and Talks
• Presentation Slides

🛠 Open-Source Libraries and Tools

• Python
• R
• Julia
• Other Languages
• AI Platforms
• Kaggle Notebooks

📚 Research: Papers and Theses

• Papers (general)
• Time Series Forecasting
• Anomaly Detection
• PhD Theses

🌍 Community, Events, and People

• Events and Workshops
• [Conferences](#conferenc