<p align="center"> <!-- <img width="230" height="230" src="docs/_static/logo.png" alt="logo"> --> <img width="1000" height="200" src="docs/_static/mammoth_banner.svg" alt="logo"> </p> <p align="center"> <img alt="GitHub commit activity" src="https://img.shields.io/github/commit-activity/m/aimagelab/mammoth"> <a href="https://aimagelab.github.io/mammoth/index.html"><img alt="Documentation" src="https://img.shields.io/badge/docs-mammoth-blue?style=flat&logo=readthedocs"></a> <img alt="GitHub stars" src="https://img.shields.io/github/stars/aimagelab/mammoth?style=social"> <img alt="PyTorch" src="https://img.shields.io/badge/PyTorch-%23EE4C2C.svg?&logo=PyTorch&logoColor=white"> </p>

🦣 Mammoth - A PyTorch Framework for Benchmarking Continual Learning

Mammoth is built to streamline the development and benchmark of continual learning research. With more than 70 methods and 20 datasets, it includes the most complete list competitors and benchmarks for research purposes.

The core idea of Mammoth is that it is designed to be modular, easy to extend, and - most importantly - easy to debug.

With Mammoth, nothing is set in stone. You can easily add new models, datasets, training strategies, or functionalities.

📖 Table of Contents

• Documentation
  • Getting Started
  • How to Run
  • How to Upgrade
  • Models Reference
  • Datasets Reference
  • Backbone Reference
  • Utils Reference
• Setup
• Examples
  • Run a model
  • Build a new model
  • Build a new dataset
• New Features
• Update Roadmap
• Models
• Datasets
• Citing the library
• On the reproducibility of Mammoth
• Contributing

📚 Documentation

<p align="center"> <a href="https://aimagelab.github.io/mammoth/"> <em style="display: inline-block; margin-top: 8px; font-size: 16px; color: #4B73C9; background-color: #f8f9fa; padding: 8px 16px; border-radius: 0 0 8px 8px; border: 1px solid #4B73C9; border-top: none; box-shadow: 0 2px 5px rgba(0,0,0,0.1);">Check out our guides on using Mammoth for continual learning research</em> <br/> <img src="https://img.shields.io/badge/Documentation-📚-4B73C9?style=for-the-badge&logo=gitbook&logoColor=white" alt="Documentation" height="40"> </a> </p>

⚙️ Setup

• 📥 Install with pip install -r requirements.txt or run it directly with uv run python main.py ...

  Note: PyTorch version >= 2.1.0 is required for scaled_dot_product_attention. If you cannot support this requirement, uncomment the lines 136-139 under scaled_dot_product_attention in backbone/vit.py.

• 🚀 Use main.py or ./utils/main.py to run experiments.
• 🧩 New models can be added to the models/ folder.
• 📊 New datasets can be added to the datasets/ folder.

🧪 Examples

Run a model

The following command will run the model derpp on the dataset seq-cifar100 with a buffer of 500 samples the some random hyperparameters for lr, alpha, and beta:

python main.py --model derpp --dataset seq-cifar100 --alpha 0.5 --beta 0.5 --lr 0.001 --buffer_size 500

To run the model with the best hyperparameters, use the --model_config=best argument:

python main.py --model derpp --dataset seq-cifar100 --model_config best

NOTE: the --model_config argument will look for a file <model_name>.yaml in the models/configs/ folder. This file should contain the hyperparameters for the best configuration of the model. You can find more information in the documentation.

Build a new model

See the documentation for a detailed guide on how to create a new model.

Build a new dataset

See the documentation for a detailed guide on how to create a new dataset.

🆕 New Features

• --loadcheck option now can load the arguments saved from the checkpoint, so you can resume the training from the last checkpoint by just running python main.py --loadcheck <checkpoint_name>.

• The training now captures the SIGINT signal (Ctrl+C) to gracefully stop the training process and save the current state. The checkpoint is saved in checkpoints/paused/ directory. This can be disabled by setting --save_after_interrupt=0 in the command line.

• Add the option --checkpoint_path to specify a custom path for saving checkpoints. By default, checkpoints are saved in the checkpoints/ directory.

• Now Mammoth can be installed with pip to be used as a library. You can install it with pip install -e . (or just uv sync) and then import it in your Python scripts. Examples of usage can be found in the examples/ directory.

  NOTE: Mammoth is not yet available on PyPI, so you need to clone the repository and run the command above to install it.

🗺️ Update Roadmap

All the code is under active development. Here are some of the features we are working on:

• 🧠 New models: We are continuously working on adding new models to the repository.
• 🔄 New training modalities: New training regimes, such a regression, segmentation, detection, etc.
• 📊 Openly accessible result dashboard: The ideal would be a dashboard to visualize the results of all the models in both their respective settings (to prove their reproducibility) and in a general setting (to compare them). This may take some time, since compute is not free.

All the new additions will try to preserve the current structure of the repository, making it easy to add new functionalities with a simple merge.

🧠 Models

Mammoth currently supports more than 70 models, with new releases covering the main competitors in literature.

<details> <summary><b>Click to expand model list</b></summary>

• AttriCLIP: A Non-Incremental Learner for Incremental Knowledge Learning (AttriCLIP): attriclip.
• Bias Correction (BiC): bic.
• CaSpeR-IL (on DER++, X-DER with RPC, iCaRL, and ER-ACE): derpp_casper, xder_rpc_casper, icarl_casper, er_ace_casper.
• CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learning (CODA-Prompt) - Requires pip install timm==0.9.8: coda-prompt.
• Continual Contrastive Interpolation Consistency (CCIC) - Requires pip install kornia: ccic.
• Continual Generative training for Incremental prompt-Learning (CGIL): cgil
• Contrastive Language-Image Pre-Training (CLIP): clip (static method with no learning).
• CSCCT (on DER++, X-DER with RPC, iCaRL, and ER-ACE): derpp_cscct, xder_rpc_cscct, icarl_cscct, er_ace_cscct.
• Dark Experience for General Continual Learning: a Strong, Simple Baseline (DER & DER++): der and derpp.
• DualPrompt: Complementary Prompting for Rehearsal-free Continual Learning (DualPrompt) - Requires pip install timm==0.9.8: dualprompt.
• Efficient Lifelong Learning with A-GEM (A-GEM, A-GEM-R - A-GEM with reservoir buffer): agem, agem_r.
• Experience Replay (ER): er.
• Experience Replay with Asymmetric Cross-Entropy (ER-ACE): er_ace.
• eXtended-DER (X-DER): xder (full version), xder_ce (X-DER with CE), xder_rpc (X-DER with RPC).
• Function Distance Regularization (FDR): fdr.
• Generating Instance-level Prompts for Rehearsal-free Continual Learning (DAP): dap.
• Gradient Episodic Memory (GEM) - Unavailable on windows: gem.
• Greedy gradient-based Sample Selection (GSS): gss.
• Greedy Sampler and Dumb Learner (GDumb): gdumb.
• Hindsight Anchor Learning (HAL): hal.
• Image-aware Decoder Enhanced à la Flamingo with Interleaved Cross-attentionS (IDEFICS): idefics (static method with no learning).
• Incremental Classifier and Representation Learning (iCaRL): icarl.
• Joint training for the General Continual setting: joint_gcl (only for General Continual).
• Large Language and Vision Assistant (LLAVA): llava (static method with no learning).
• Learning a Unified Classifier Incrementally via Rebalancing (LUCIR): lucir.
• Learning to Prompt (L2P) - Requires pip install timm==0.9.8: l2p.
• Learning without Forgetting (LwF): lwf.
• Learning without Forgetting adapted for Multi-Class classification (LwF.MC): lwf_mc (from the iCaRL paper).
• Learning without Shortcuts (LwS): lws.
• LiDER (on DER++, iCaRL, GDumb, and ER-ACE): derpp_lider, icarl_lider, gdumb_lider, er_ace_lider.
• May the Forgetting Be with You: Alternate Replay for Learning with Noisy Labels (AER & ABS): er_ace_aer_abs.
• Meta-Experience Replay (MER): mer.
• Mixture-of-Experts Adapters (MoE Adapters): moe_adapters.
• Online Continual Learning on a Contaminated Data Stream with Blurry Task Boundaries (PuriDivER): puridiver.
• online Elastic Weight Consolidation (oEWC): ewc_on.
• Progressive Neural Networks (PNN): pnn.
• Random Projections and Pre-trained Models for Continual Learning (RanPAC): ranpac.
• Regular Polytope Classifier (RPC): rpc.
• Rethinking Experience Replay: a Bag of Tricks for Continual Learning (ER-ACE with tricks): er_ace_tricks.
• Semantic Two-level Additive Residual Prompt (STAR-Prompt): starprompt. Also includes the first-stage only (first_stage_starprompt) and second-stage only (second_stage_starprompt) versions.
• SLCA: Slow Learner with Classifier Alig