Advanced AI explainability for PyTorch

pip install grad-cam

Documentation with advanced tutorials: https://jacobgil.github.io/pytorch-gradcam-book

This is a package with state of the art methods for Explainable AI for computer vision. This can be used for diagnosing model predictions, either in production or while developing models. The aim is also to serve as a benchmark of algorithms and metrics for research of new explainability methods.

⭐ Comprehensive collection of Pixel Attribution methods for Computer Vision.

⭐ Tested on many Common CNN Networks and Vision Transformers.

⭐ Advanced use cases: Works with Classification, Object Detection, Semantic Segmentation, Embedding-similarity and more.

⭐ Includes smoothing methods to make the CAMs look nice.

⭐ High performance: full support for batches of images in all methods.

⭐ Includes metrics for checking if you can trust the explanations, and tuning them for best performance.

| Method | What it does | |---------------------|-----------------------------------------------------------------------------------------------------------------------------| | GradCAM | Weight the 2D activations by the average gradient | | HiResCAM | Like GradCAM but element-wise multiply the activations with the gradients; provably guaranteed faithfulness for certain models | | GradCAMElementWise | Like GradCAM but element-wise multiply the activations with the gradients then apply a ReLU operation before summing | | GradCAM++ | Like GradCAM but uses second order gradients | | XGradCAM | Like GradCAM but scale the gradients by the normalized activations | | AblationCAM | Zero out activations and measure how the output drops (this repository includes a fast batched implementation) | | ScoreCAM | Perbutate the image by the scaled activations and measure how the output drops | | EigenCAM | Takes the first principle component of the 2D Activations (no class discrimination, but seems to give great results) | | EigenGradCAM | Like EigenCAM but with class discrimination: First principle component of Activations*Grad. Looks like GradCAM, but cleaner | | LayerCAM | Spatially weight the activations by positive gradients. Works better especially in lower layers | | FullGrad | Computes the gradients of the biases from all over the network, and then sums them | | Deep Feature Factorizations | Non Negative Matrix Factorization on the 2D activations | | KPCA-CAM | Like EigenCAM but with Kernel PCA instead of PCA |
| FEM | A gradient free method that binarizes activations by an activation > mean + k * std rule. | | ShapleyCAM | Weight the activations using the gradient and Hessian-vector product.| | FinerCAM | Improves fine-grained classification by comparing similar classes, suppressing shared features and highlighting discriminative details. | | SegEigenCAM | Like EigenCAM but with gradient weighting (absolute gradients ⊙ activations) before SVD and sign correction to fix SVD sign ambiguity; designed for semantic segmentation | | RefineCAM | A meta-method that computes a CAM at multiple layers, and then it combines them to ubtain a higher resolution and better focused CAM. It can be used with any of the other CAM methods. |

Visual Examples

| What makes the network think the image label is 'pug, pug-dog' | What makes the network think the image label is 'tabby, tabby cat' | Combining Grad-CAM with Guided Backpropagation for the 'pug, pug-dog' class | | ---------------------------------------------------------------|--------------------|-----------------------------------------------------------------------------| <img src="https://github.com/jacobgil/pytorch-grad-cam/blob/master/examples/dog.jpg?raw=true" width="256" height="256"> | <img src="https://github.com/jacobgil/pytorch-grad-cam/blob/master/examples/cat.jpg?raw=true" width="256" height="256"> | <img src="https://github.com/jacobgil/pytorch-grad-cam/blob/master/examples/cam_gb_dog.jpg?raw=true" width="256" height="256"> |

Object Detection and Semantic Segmentation

| Object Detection | Semantic Segmentation | | -----------------|-----------------------| | <img src="./examples/both_detection.png" width="256" height="256"> | <img src="./examples/cars_segmentation.png" width="256" height="200"> |

| 3D Medical Semantic Segmentation | | -------------------------- | | <img src="./examples/multiorgan_segmentation.gif" width="539">|

Explaining similarity to other images / embeddings

<img src="./examples/embeddings.png">

Deep Feature Factorization

<img src="./examples/dff1.png"> <img src="./examples/dff2.png">

CLIP

| Explaining the text prompt "a dog" | Explaining the text prompt "a cat" | | -----------------------------------|------------------------------------| <img src="https://github.com/jacobgil/pytorch-grad-cam/blob/master/examples/clip_dog.jpg?raw=true" width="256" height="256"> | <img src="https://github.com/jacobgil/pytorch-grad-cam/blob/master/examples/clip_cat.jpg?raw=true" width="256" height="256"> |

Classification

Resnet50:

| Category | Image | GradCAM | AblationCAM | ScoreCAM | | ---------|-------|----------|------------|------------| | Dog | | | | | | Cat | | | | |

Vision Transfomer (Deit Tiny):

| Category | Image | GradCAM | AblationCAM | ScoreCAM | | ---------|-------|----------|------------|------------| | Dog | | | | | | Cat | | | | |

Swin Transfomer (Tiny window:7 patch:4 input-size:224):

| Category | Image | GradCAM | AblationCAM | ScoreCAM | | ---------|-------|----------|------------|------------| | Dog | | | | | | Cat | | | | |

Metrics and Evaluation for XAI

<img src="./examples/metrics.png"> <img src="./examples/road.png">

---

Usage examples

from pytorch_grad_cam import GradCAM, HiResCAM, ScoreCAM, GradCAMPlusPlus, AblationCAM, XGradCAM, EigenCAM, FullGrad
from pytorch_grad_cam.utils.model_targets import ClassifierOutputTarget
from pytorch_grad_cam.utils.image import show_cam_on_image
from torchvision.models import resnet50, ResNet50_Weights

model = resnet50(weights=ResNet50_Weights.DEFAULT)
target_layers = [model.layer4[-1]]
input_tensor = # Create an input tensor image for your model..
# Note: input_tensor can be a batch tensor with several images!

# We have to specify the target we want to generate the CAM for.
targets = [ClassifierOutputTarget(281)]

# Construct the CAM object once, and then re-use it on many images.
with GradCAM(model=model, target_layers=target_layers) as cam:
  # You can also pass aug_smooth=True and eigen_smooth=True, to apply smoothing.
  grayscale_cam = cam(input_tensor=input_tensor, targets=targets)
  # In this example grayscale_cam has only one image in the batch:
  grayscale_cam = grayscale_cam[0, :]
  visualization = show_cam_on_image(rgb_img, grayscale_cam, use_rgb=True)
  # You can also get the model outputs without having to redo inference
  model_outputs = cam.outputs

cam.py has a more detailed usage example.

---

Choosing the layer(s) to extract activations from

You need to choose the target layer to compute the CAM for. Some common choices are:

• FasterRCNN: model.backbone
• Resnet18 and 50: model.layer4[-1]
• VGG, densenet161 and mobilenet: model.features[-1]
• mnasnet1_0: model.layers[-1]
• ViT: