DARE

This is the official repository of the ICML, 2024 paper "Gradual Divergence for Seamless Adaptation: A Novel Domain Incremental Learning Method" by Kishaan Jeeveswaran, Elahe Arani and Bahram Zonooz.

We extended the Mammoth framework with our method (DARE/DARE++)

TLDR: A novel representation rehearsal-based domain incremental learning approach that, by incorporating a three-stage methodology to mitigate representation drift and regularization, enables effective and memory-efficient continual learning.

Schematic of MTSL:

Setup:

OUTPUT_DIR: Directory to save output contents.<br /> Update the base_data_path() in utils/conf.py with the folder where the dataset is stored.<br />

Datasets supported: <br />

• iCIFAR_20
• DN4IL

DARE Training Script:

To train DARE on iCifar-20 dataset, 5 tasks, with buffer size 50:

python main.py --model maxd_ema --dataset super-cifar --img_size 32  --num_tasks 5 --alpha 0.3 --beta 0.1 --maximize_task hcr --maxd_weight 0.1  --mind_weight 1  --logitb_weight 1 --logitc_weight 1 --iterative_buffer --supcon_weight 0.05 --supcon_temp 1.2 --frozen_supcon --intermediate_sampling --std 4 --reduce_lr --each_epoch --buffer_size 50 --lr 0.04 --batch_size 32 --minibatch_size 32 --n_epochs 50 results.csv --output_folder <OUTPUT_DIR> --tensorboard

To train DARE on DN4IL dataset, 6 tasks, with buffer size 50:

python main.py --model maxd_ema --dataset domain-net --img_size 64 --num_tasks 6 --alpha 0.1 --beta 0.2 --maximize_task hcr --maxd_weight 0.1  --mind_weight 1  --logitb_weight 1 --logitc_weight 1 --iterative_buffer --supcon_weight 0.05 --supcon_temp 0.8 --frozen_supcon --intermediate_sampling --std 4 --reduce_lr --each_epoch --buffer_size 200 --lr 0.04 --batch_size 32 --minibatch_size 32 --n_epochs 50 --output_folder <OUTPUT_DIR> --tensorboard

Cite Our Work:

If you find the code useful in your research please consider citing our paper.

<pre> @misc{jeeveswaran2024gradualdivergenceseamlessadaptation, title={Gradual Divergence for Seamless Adaptation: A Novel Domain Incremental Learning Method}, author={Kishaan Jeeveswaran and Elahe Arani and Bahram Zonooz}, year={2024}, eprint={2406.16231}, archivePrefix={arXiv}, primaryClass={cs.LG}, url={https://arxiv.org/abs/2406.16231}, } </pre>