Blades
⚔️ Blades: A Unified Benchmark Suite for Attacks and Defenses in Federated Learning
Install / Use
/learn @lishenghui/BladesREADME
.. raw:: html
<div style="text-align: center;"> <a href="https://arxiv.org/pdf/2206.05359.pdf"> <img alt="Tests Status" src="https://img.shields.io/badge/arXiv-2206.05359-red?logo=arxiv&style=flat-square&link=https%3A%2F%2Farxiv.org%2Fpdf%2F2206.05359.pdf"/> </a> <a href="https://github.com/lishenghui/blades"> <img alt="Build Status" src="https://img.shields.io/github/last-commit/lishenghui/blades/master?logo=Github"/> </a> <a href="https://github.com/lishenghui/blades/actions/workflows/unit-tests.yml"> <img alt="Tests Status" src="https://github.com/lishenghui/blades/actions/workflows/unit-tests.yml/badge.svg?branch=master"/> </a> <a href="https://blades.readthedocs.io/en/latest/?badge=latest"> <img alt="Docs" src="https://readthedocs.org/projects/blades/badge/?version=latest"/> </a> <a href="https://pytorch.org/get-started/pytorch-2.0/"> <img alt="Docs" src="https://img.shields.io/badge/Pytorch-2.0-brightgreen?logo=pytorch&logoColor=red"/> </a> <a href="https://docs.ray.io/en/releases-2.9.0/"> <img alt="Docs" src="https://img.shields.io/badge/Ray-2.9-brightgreen?logo=ray&logoColor=blue"/> </a> <a href="https://github.com/lishenghui/blades/blob/master/LICENSE"> <img alt="License" src="https://img.shields.io/github/license/lishenghui/blades?logo=apache&logoColor=red"/> </a> </div>.. .. image:: https://img.shields.io/github/last-commit/lishenghui/blades/master?logo=Github :alt: GitHub last commit (branch) :target: https://github.com/lishenghui/blades .. image:: https://github.com/lishenghui/blades/actions/workflows/unit-tests.yml/badge.svg?branch=master :alt: GitHub Workflow Status (with event)
.. container:: badges
.. image:: https://img.shields.io/github/last-commit/lishenghui/blades/master?logo=Github
:alt: GitHub last commit (branch)
:target: https://github.com/lishenghui/blades
.. image:: https://github.com/lishenghui/blades/actions/workflows/unit-tests.yml/badge.svg?branch=master
:alt: GitHub Workflow Status (with event)
.. image:: https://img.shields.io/badge/Pytorch-2.0-brightgreen?logo=pytorch&logoColor=red
:alt: Static Badge
:target: https://pytorch.org/get-started/pytorch-2.0/
.. image:: https://img.shields.io/badge/Ray-2.8-brightgreen?logo=ray&logoColor=blue
:alt: Static Badge
:target: https://docs.ray.io/en/releases-2.8.0/
.. image:: https://readthedocs.org/projects/blades/badge/?version=latest
:target: https://blades.readthedocs.io/en/latest/?badge=latest
:alt: Documentation Status
.. image:: https://img.shields.io/github/license/lishenghui/blades?logo=apache&logoColor=red
:alt: GitHub
:target: https://github.com/lishenghui/blades/blob/master/LICENSE
.. image:: https://img.shields.io/badge/arXiv-2206.05359-red?logo=arxiv&style=flat-square&link=https%3A%2F%2Farxiv.org%2Fpdf%2F2206.05359.pdf
:alt: Static Badge
:target: https://arxiv.org/pdf/2206.05359.pdf
.. raw:: html
<p align=center> <img src="https://github.com/lishenghui/blades/blob/master/docs/source/images/client_pipeline.png" width="1000" alt="Blades Logo"> </p>Installation
.. code-block:: bash
git clone https://github.com/lishenghui/blades
cd blades
pip install -v -e .
# "-v" means verbose, or more output
# "-e" means installing a project in editable mode,
# thus any local modifications made to the code will take effect without reinstallation.
.. code-block:: bash
cd blades/blades
python train.py file ./tuned_examples/fedsgd_cnn_fashion_mnist.yaml
Blades internally calls ray.tune <https://docs.ray.io/en/latest/tune/tutorials/tune-output.html>_; therefore, the experimental results are output to its default directory: ~/ray_results.
Experiment Results
.. image:: https://github.com/lishenghui/blades/blob/master/docs/source/images/fashion_mnist.png
.. image:: https://github.com/lishenghui/blades/blob/master/docs/source/images/cifar10.png
Cluster Deployment
To run blades on a cluster, you only need to deploy Ray cluster according to the official guide <https://docs.ray.io/en/latest/cluster/user-guide.html>_.
Built-in Implementations
In detail, the following strategies are currently implemented:
Attacks
General Attacks
^^^^^^^^^^^^^^^^^
+--------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------+
| Strategy | Description | Sourse |
+====================+==========================================================================================================================================================================================================+===========================================================================================================+
| Noise | Put random noise to the updates. | Sourse <https://github.com/lishenghui/blades/blob/master/blades/adversaries/noise_adversary.py>_ |
+--------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------+
| Labelflipping | Fang et al. Local Model Poisoning Attacks to Byzantine-Robust Federated Learning <https://www.usenix.org/conference/usenixsecurity20/presentation/fang>, USENIX Security' 20 | Sourse <https://github.com/lishenghui/blades/blob/master/blades/adversaries/labelflip_adversary.py> |
+--------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------+
| Signflipping | Li et al. RSA: Byzantine-Robust Stochastic Aggregation Methods for Distributed Learning from Heterogeneous Datasets <https://ojs.aaai.org/index.php/AAAI/article/view/3968>, AAAI' 19 | Sourse <https://github.com/lishenghui/blades/blob/master/blades/adversaries/signflip_adversary.py> |
+--------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------+
| ALIE | Baruch et al. A little is enough: Circumventing defenses for distributed learning <https://proceedings.neurips.cc/paper/2019/hash/ec1c59141046cd1866bbbcdfb6ae31d4-Abstract.html>_ NeurIPS' 19 | Sourse <https://github.com/lishenghui/blades/blob/master/blades/adversaries/alie_adversary.py>_ |
+--------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------+
| IPM | Xie et al. Fall of empires: Breaking byzantine- tolerant sgd by inner product manipulation <https://arxiv.org/abs/1903.03936>, UAI' 20 | Sourse <https://github.com/lishenghui/blades/blob/master/blades/adversaries/ipm_adversary.py> |
+--------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------+
Adaptive Attacks ^^^^^^^^^^^^^^^^^ +--------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------+ | Strategy | Description | Sourse | +==========================+=====================================================================================================================================================================================+================================================================
Related Skills
YC-Killer
2.7kA library of enterprise-grade AI agents designed to democratize artificial intelligence and provide free, open-source alternatives to overvalued Y Combinator startups. If you are excited about democratizing AI access & AI agents, please star ⭐️ this repository and use the link in the readme to join our open source AI research team.
best-practices-researcher
The most comprehensive Claude Code skills registry | Web Search: https://skills-registry-web.vercel.app
groundhog
398Groundhog's primary purpose is to teach people how Cursor and all these other coding agents work under the hood. If you understand how these coding assistants work from first principles, then you can drive these tools harder (or perhaps make your own!).
isf-agent
a repo for an agent that helps researchers apply for isf funding
