DWSNets

Official implementation for Equivariant Architectures for Learning in Deep Weight Spaces by Aviv Navon, Aviv Shamsian, Idan Achituve, Ethan Fetaya, Gal Chechik, Haggai Maron.

<p align="center"> <img src=misc/sym.png height="400"> </p>

Our implementation follows the block structure as described in the paper.

Setup environment

To run the experiments, first create clean virtual environment and install the requirements.

conda create -n dwsnets python=3.9
conda activate dwsnets
conda install pytorch==1.12.1 torchvision==0.13.1 torchaudio==0.12.1 cudatoolkit=11.3 -c pytorch

Install the repo:

git clone https://github.com/AvivNavon/DWSNets.git
cd DWSNets
pip install -e .

Introduction Notebook

An introduction notebook for INR classification with DWSNets:

Run experiment

To run specific experiment, please follow the instructions in the README file within each experiment folder. It provides full instructions and details for downloading the data and reproducing the results reported in the paper.

Dataset

The datasets are available here.

• Sine regression INRs
• MNIST INRs
• SSL INRs
• CIFAR10 classifiers

Citation

If you find our work or this code to be useful in your own research, please consider citing the following paper:

@InProceedings{pmlr-v202-navon23a,
  title = 	 {Equivariant Architectures for Learning in Deep Weight Spaces},
  author =       {Navon, Aviv and Shamsian, Aviv and Achituve, Idan and Fetaya, Ethan and Chechik, Gal and Maron, Haggai},
  booktitle = 	 {Proceedings of the 40th International Conference on Machine Learning},
  pages = 	 {25790--25816},
  year = 	 {2023},
  editor = 	 {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan},
  volume = 	 {202},
  series = 	 {Proceedings of Machine Learning Research},
  month = 	 {23--29 Jul},
  publisher =    {PMLR},
  pdf = 	 {https://proceedings.mlr.press/v202/navon23a/navon23a.pdf},
  url = 	 {https://proceedings.mlr.press/v202/navon23a.html},
}