ABL
Anti-Backdoor learning (NeurIPS 2021)
Install / Use
/learn @bboylyg/ABLREADME
Anti-Backdoor Learning
PyTorch Code for NeurIPS 2021 paper Anti-Backdoor Learning: Training Clean Models on Poisoned Data.
The Anti-Backdoor Learning Leaderboard
We encourage submissions of anti-backdoor learning methods to our leaderboard.
Evaluation：We will run the submitted learning method on poisoned CIFAR-10 datasets by 10 backdoor attacks used in our paper, then test the Attack Sucess Rate (ASR) and Clean Accuracy (CA) of the final model.
Update: This leaderboard is created on 2021/10/21 and updated on 021/10/21.
| # | Paper | Venue | Poisoned data | Architecture | Attack | ASR (%)| CA (%)| | ----- | :------------------------: | :----------: | :------------: | :----------: | :---------: | :-----------: | :----------: | | 1 | ABL | NeurIPS 2021 | available | WRN-16-1 | BadNets | 3.04 | 86.11 | | 2 | | | | | | | | | 3 | | | | | | | | | 4 | | | | | | | | | 5 | | | | | | | | | 6 | | | | | | | | | 7 | | | | | | | | | 8 | | | | | | | |
Verifying the unlearning effect of ABL with 1% isolated data:
An example with a pretrained model
WRN-16-1, CIFAR-10, GridTrigger, target label 0, weights: ./weight/backdoored_model.
Run the following command to verify the unlearning effect:
$ python quick_unlearning_demo.py
The training logs are shown below. 1% isolation = 500 images from poisoned CIFAR-10. It shows the ASR (bad acc) of drops from 99.99% to 0.48% with no obviouse drop of clean acc.
Epoch,Test_clean_acc,Test_bad_acc,Test_clean_loss,Test_bad_loss
0,82.77777777777777,99.9888888888889,0.9145596397187975,0.0007119161817762587
Epoch,Test_clean_acc,Test_bad_acc,Test_clean_loss,Test_bad_loss
1,82.97777777777777,47.13333333333333,0.9546798907385932,4.189897534688313
Epoch,Test_clean_acc,Test_bad_acc,Test_clean_loss,Test_bad_loss
2,82.46666666666667,5.766666666666667,1.034722186088562,15.361101960923937
Epoch,Test_clean_acc,Test_bad_acc,Test_clean_loss,Test_bad_loss
3,82.15555555555555,1.5222222222222221,1.0855470676422119,22.175255742390952
Epoch,Test_clean_acc,Test_bad_acc,Test_clean_loss,Test_bad_loss
4,82.0111111111111,0.7111111111111111,1.1183592330084906,26.754894670274524
Epoch,Test_clean_acc,Test_bad_acc,Test_clean_loss,Test_bad_loss
5,81.86666666666666,0.4777777777777778,1.1441074348025853,30.429284422132703
The unlearned model will be saved to 'weight/ABL_results/<model_name>.tar'
Please read quick_unlearning_demo.py to adjust the default parameters for your experiment.
More defense results on BadNets model trained with Data Augmentation
[Logs for our ABL against Badnet Attacks]
----------- Model Fine-tuning --------------
epoch: 40 lr: 0.0100
Epoch[41]:[200/774] loss:0.1456(0.1240) prec@1:98.44(95.84) prec@5:98.44(99.96)
Epoch[41]:[400/774] loss:0.0553(0.1080) prec@1:98.44(96.38) prec@5:100.00(99.97)
Epoch[41]:[600/774] loss:0.0693(0.1015) prec@1:96.88(96.63) prec@5:100.00(99.97)
[Clean] Prec@1: 92.23, Loss: 0.2408
[Bad] Prec@1: 100.00, Loss: 0.0001
epoch: 41 lr: 0.0100
Epoch[42]:[200/774] loss:0.0532(0.0653) prec@1:98.44(97.89) prec@5:100.00(100.00)
Epoch[42]:[400/774] loss:0.0534(0.0659) prec@1:98.44(97.76) prec@5:100.00(100.00)
Epoch[42]:[600/774] loss:0.0514(0.0659) prec@1:96.88(97.76) prec@5:100.00(99.99)
[Clean] Prec@1: 92.60, Loss: 0.2390
[Bad] Prec@1: 100.00, Loss: 0.0000
epoch: 42 lr: 0.0100
Epoch[43]:[200/774] loss:0.0054(0.0499) prec@1:100.00(98.33) prec@5:100.00(99.99)
Epoch[43]:[400/774] loss:0.0429(0.0525) prec@1:98.44(98.21) prec@5:100.00(99.99)
Epoch[43]:[600/774] loss:0.0448(0.0537) prec@1:98.44(98.19) prec@5:100.00(99.99)
[Clean] Prec@1: 92.52, Loss: 0.2409
[Bad] Prec@1: 100.00, Loss: 0.0001
epoch: 43 lr: 0.0100
Epoch[44]:[200/774] loss:0.0253(0.0472) prec@1:98.44(98.41) prec@5:100.00(99.99)
Epoch[44]:[400/774] loss:0.0104(0.0463) prec@1:100.00(98.43) prec@5:100.00(99.99)
Epoch[44]:[600/774] loss:0.0200(0.0452) prec@1:100.00(98.46) prec@5:100.00(99.99)
[Clean] Prec@1: 92.60, Loss: 0.2459
[Bad] Prec@1: 100.00, Loss: 0.0000
epoch: 44 lr: 0.0100
Epoch[45]:[200/774] loss:0.0510(0.0385) prec@1:98.44(98.79) prec@5:100.00(99.99)
Epoch[45]:[400/774] loss:0.0244(0.0381) prec@1:98.44(98.82) prec@5:100.00(100.00)
Epoch[45]:[600/774] loss:0.0203(0.0391) prec@1:100.00(98.83) prec@5:100.00(99.99)
[Clean] Prec@1: 92.81, Loss: 0.2484
[Bad] Prec@1: 100.00, Loss: 0.0000
epoch: 45 lr: 0.0100
Epoch[46]:[200/774] loss:0.0110(0.0374) prec@1:100.00(98.75) prec@5:100.00(99.99)
Epoch[46]:[400/774] loss:0.0204(0.0371) prec@1:98.44(98.79) prec@5:100.00(99.99)
Epoch[46]:[600/774] loss:0.0183(0.0369) prec@1:100.00(98.76) prec@5:100.00(99.99)
[Clean] Prec@1: 92.99, Loss: 0.2495
[Bad] Prec@1: 100.00, Loss: 0.0000
epoch: 46 lr: 0.0100
Epoch[47]:[200/774] loss:0.0452(0.0315) prec@1:98.44(98.97) prec@5:100.00(100.00)
Epoch[47]:[400/774] loss:0.0315(0.0310) prec@1:98.44(98.98) prec@5:100.00(100.00)
Epoch[47]:[600/774] loss:0.0298(0.0303) prec@1:100.00(99.01) prec@5:100.00(100.00)
[Clean] Prec@1: 92.82, Loss: 0.2563
[Bad] Prec@1: 100.00, Loss: 0.0000
epoch: 47 lr: 0.0100
Epoch[48]:[200/774] loss:0.0397(0.0269) prec@1:98.44(99.12) prec@5:100.00(100.00)
Epoch[48]:[400/774] loss:0.0617(0.0262) prec@1:98.44(99.16) prec@5:100.00(100.00)
Epoch[48]:[600/774] loss:0.0630(0.0270) prec@1:98.44(99.16) prec@5:100.00(100.00)
[Clean] Prec@1: 92.81, Loss: 0.2678
[Bad] Prec@1: 100.00, Loss: 0.0000
epoch: 48 lr: 0.0100
Epoch[49]:[200/774] loss:0.0251(0.0267) prec@1:100.00(99.15) prec@5:100.00(100.00)
Epoch[49]:[400/774] loss:0.0298(0.0262) prec@1:98.44(99.14) prec@5:100.00(100.00)
Epoch[49]:[600/774] loss:0.0384(0.0258) prec@1:98.44(99.15) prec@5:100.00(100.00)
[Clean] Prec@1: 93.09, Loss: 0.2586
[Bad] Prec@1: 100.00, Loss: 0.0002
epoch: 49 lr: 0.0100
Epoch[50]:[200/774] loss:0.0359(0.0203) prec@1:98.44(99.30) prec@5:100.00(100.00)
Epoch[50]:[400/774] loss:0.0062(0.0214) prec@1:100.00(99.27) prec@5:100.00(100.00)
Epoch[50]:[600/774] loss:0.0418(0.0222) prec@1:98.44(99.25) prec@5:100.00(100.00)
[Clean] Prec@1: 93.03, Loss: 0.2626
[Bad] Prec@1: 100.00, Loss: 0.0001
epoch: 50 lr: 0.0100
Epoch[51]:[200/774] loss:0.0040(0.0222) prec@1:100.00(99.27) prec@5:100.00(100.00)
Epoch[51]:[400/774] loss:0.0269(0.0236) prec@1:98.44(99.21) prec@5:100.00(100.00)
Epoch[51]:[600/774] loss:0.0219(0.0234) prec@1:100.00(99.23) prec@5:100.00(100.00)
[Clean] Prec@1: 93.19, Loss: 0.2604
[Bad] Prec@1: 100.00, Loss: 0.0000
epoch: 51 lr: 0.0100
Epoch[52]:[200/774] loss:0.0154(0.0201) prec@1:98.44(99.34) prec@5:100.00(100.00)
Epoch[52]:[400/774] loss:0.0328(0.0200) prec@1:98.44(99.38) prec@5:100.00(100.00)
Epoch[52]:[600/774] loss:0.0220(0.0204) prec@1:98.44(99.36) prec@5:100.00(100.00)
[Clean] Prec@1: 93.27, Loss: 0.2652
[Bad] Prec@1: 100.00, Loss: 0.0000
epoch: 52 lr: 0.0100
Epoch[53]:[200/774] loss:0.0090(0.0194) prec@1:100.00(99.39) prec@5:100.00(100.00)
Epoch[53]:[400/774] loss:0.0019(0.0195) prec@1:100.00(99.41) prec@5:100.00(100.00)
Epoch[53]:[600/774] loss:0.0402(0.0190) prec@1:98.44(99.45) prec@5:100.00(100.00)
[Clean] Prec@1: 93.04, Loss: 0.2735
[Bad] Prec@1: 100.00, Loss: 0.0000
epoch: 53 lr: 0.0100
Epoch[54]:[200/774] loss:0.0154(0.0186) prec@1:100.00(99.38) prec@5:100.00(100.00)
Epoch[54]:[400/774] loss:0.0124(0.0182) prec@1:100.00(99.40) prec@5:100.00(100.00)
Epoch[54]:[600/774] loss:0.0144(0.0181) prec@1:100.00(99.45) prec@5:100.00(100.00)
[Clean] Prec@1: 93.17, Loss: 0.2693
[Bad] Prec@1: 100.00, Loss: 0.0000
epoch: 54 lr: 0.0100
Epoch[55]:[200/774] loss:0.0119(0.0168) prec@1:100.00(99.43) prec@5:100.00(100.00)
Epoch[55]:[400/774] loss:0.0228(0.0170) prec@1:98.44(99.42) prec@5:100.00(100.00)
Epoch[55]:[600/774] loss:0.0096(0.0164) prec@1:100.00(99.47) prec@5:100.00(100.00)
[Clean] Prec@1: 92.84, Loss: 0.2786
[Bad] Prec@1: 100.00, Loss: 0.0001
epoch: 55 lr: 0.0100
Epoch[56]:[200/774] loss:0.0307(0.0146) prec@1:98.44(99.51) prec@5:100.00(100.00)
Epoch[56]:[400/774] loss:0.0065(0.0149) prec@1:100.00(99.52) prec@5:100.00(100.00)
Epoch[56]:[600/774] loss:0.0348(0.0155) prec@1:98.44(99.50) prec@5:100.00(100.00)
[Clean] Prec@1: 93.12, Loss: 0.2794
[Bad] Prec@1: 100.00, Loss: 0.0000
epoch: 56 lr: 0.0100
Epoch[57]:[200/774] loss:0.0014(0.0134) prec@1:100.00(99.59) prec@5:100.00(100.00)
Epoch[57]:[400/774] loss:0.0060(0.0133) prec@1:100.00(99.59) prec@5:100.00(100.00)
Epoch[57]:[600/774] loss:0.0400(0.0133) prec@1:95.31(99.61) prec@5:100.00(100.00)
[Clean] Prec@1: 93.13, Loss: 0.2819
[Bad] Prec@1: 100.00, Loss: 0.0000
epoch: 57 lr: 0.0100
Epoch[58]:[200/774] loss:0.0062(0.0122) prec@1:100.00(99.60) prec@5:100.00(100.00)
Epoch[58]:[400/774] loss:0.0065(0.0134) prec@1:100.00(99.56) prec@5:100.00(100.00)
Epoch[58]:[600/774] loss:0
Related Skills
YC-Killer
2.7kA library of enterprise-grade AI agents designed to democratize artificial intelligence and provide free, open-source alternatives to overvalued Y Combinator startups. If you are excited about democratizing AI access & AI agents, please star ⭐️ this repository and use the link in the readme to join our open source AI research team.
groundhog
401Groundhog's primary purpose is to teach people how Cursor and all these other coding agents work under the hood. If you understand how these coding assistants work from first principles, then you can drive these tools harder (or perhaps make your own!).
last30days-skill
20.5kAI agent skill that researches any topic across Reddit, X, YouTube, HN, Polymarket, and the web - then synthesizes a grounded summary
000-main-rules
Project Context - Name: Interactive Developer Portfolio - Stack: Next.js (App Router), TypeScript, React, Tailwind CSS, Three.js - Architecture: Component-driven UI with a strict separation of conce
