Generalizing Denoising to Non-Equilibrium Structures Improves Equivariant Force Fields

Paper | OpenReview

This repository contains the official PyTorch implementation of the work "Generalizing Denoising to Non-Equilibrium Structures Improves Equivariant Force Fields" (TMLR 2024). We show that force encoding enables generalizing denoising to non-equilibrium structures and propose to use DeNS (Denoising Non-Equilibrium Structures) as an auxiliary task to improve the performance on energy and force predictions.

We provide the code for training EquiformerV2 with DeNS on OC20 and OC22 datasets here and training Equiformer with DeNS on MD17 in this repository.

<p align="center"> <img src="fig/denoising_structures_overview.png" alt="photo not available" width="98%" height="98%"> </p> <p align="center"> <img src="fig/dens_training_process.png" alt="photo not available" width="98%" height="98%"> </p> <p align="center"> <img src="fig/dens_oc20_all+md.png" alt="photo not available" width="98%" height="98%"> </p> <p align="center"> <img src="fig/dens_oc20_leaderboard.png" alt="photo not available" width="98%" height="98%"> </p> <p align="center"> <img src="fig/dens_oc22.png" alt="photo not available" width="98%" height="98%"> </p> <p align="center"> <img src="fig/dens_md17.png" alt="photo not available" width="98%" height="98%"> </p>

As demonstrated in OMat24 paper, EquiformerV2 + DeNS achieves state-of-the-art results on Matbench Discovery leaderboard as of October 18, 2024.

<p align="center"> <img src="fig/equiformer_v2_dens_matbench_discovery.png" alt="photo not available" width="98%" height="98%"> </p>

Content

0. Environment Setup
1. File Structure
2. Training
3. Checkpoint
4. Evaluation
5. Citation
6. Acknowledgement

Environment Setup

Environment

See here for setting up the environment.

OC20

Please first set up the environment and file structures (placing this repository under ocp and rename it to experimental) following the above Environment section.

The OC20 S2EF dataset can be downloaded by following instructions in their GitHub repository.

For example, we can download the OC20 S2EF-2M dataset by running:

    cd ocp
    python scripts/download_data.py --task s2ef --split "2M" --num-workers 8 --ref-energy

We also need to download the "val_id" data split to run training.

After downloading, the datasets should be under ocp/data.

To train on different splits like All and All+MD, we can follow the same link above to download the datasets.

OC22

Please first set up the environment and file structures (placing this repository under ocp and rename it to experimental) following the above Environment section.

Similar to OC20, the OC22 dataset can be downloaded by following instructions in their GitHub repository.

MD17

Please refer to this repository for training Equiformer with DeNS on MD17.

File Structure

1. configs contains config files for training with DeNS on different datasets.
2. datasets contains LMDB dataset class that can distinguish whether structures in OC20 come from All split or MD split.
3. model contains EquiformerV2 and eSCN models capable of training with DeNS.
4. scripts contains the scripts for launching training based on config files.
5. trainers contains the code for training models for S2EF and with DeNS.

Training

OC20

1. Modify the paths to datasets before launching training. For example, we need to modify the path to the training set as here and the validation set as here before training EquiformerV2 with DeNS on OC20 S2EF-2M dataset for 12 epochs.

2. We train EquiformerV2 with DeNS on the OC20 S2EF-2M dataset for 12 epochs by running:

       cd ocp/
       sh experimental/scripts/train/oc20/s2ef/equiformer_v2/equiformer_dens_v2_N@12_L@6_M@2_epochs@12_splits@2M_g@multi-nodes.sh
   

   Note that following the above Environment section, we will run the script under ocp. This script will use 2 nodes with 8 GPUs on each node.

   We can also run training on 8 GPUs on 1 node:

       cd ocp/
       sh experimental/scripts/train/oc20/s2ef/equiformer_v2/equiformer_dens_v2_N@12_L@6_M@2_epochs@12_splits@2M_g@8.sh
   

   Note that this is to show that we can train on a single node and the results are not the same as training on 16 GPUs.

   Similarly, we train EquiformerV2 with DeNS on the OC20 S2EF-2M dataset for 30 epochs by running:

       cd ocp/
       sh experimental/scripts/train/oc20/s2ef/equiformer_v2/equiformer_dens_v2_N@12_L@6_M@2_epochs@30_splits@2M_g@multi-nodes.sh
   

   This script will use 4 nodes with 8 GPUs on each node.

3. We train EquiformerV2 with DeNS on the OC20 S2EF-All+MD dataset by running:

       cd ocp/
       sh experimental/scripts/train/oc20/s2ef/equiformer_v2/equiformer_dens_v2_N@20_L@6_M@3_splits@all-md_g@multi-nodes.sh
   

   This script will use 16 nodes with 8 GPUs on each node.

   We use a slightly different dataset class DeNSLmdbDataset so that we can differentiate whether a structure is from the All split or the MD split. This corresponds to the code here and requires relaxations and md to exist in data_log.*.txt files under the All+MD data directory. Those data_log.*.txt should look like:

       # for All split
       /.../relaxations/.../random1331004.traj,258,365
       ...
   

   After reading the lmdb files, the DeNSLmdbDataset dataset will add a new attribute md as here.

OC22

1. Modify the paths to datasets before launching training. Specifically, we need to modify the path to the training set as here and the validation set as here.

   In addition, we need to download the linear reference file from here and then add the path to the linear reference file as here and here.

   Finally, we download the OC20 reference information file from here and add the path to that file as here and here.

2. We train EquiformerV2 with DeNS on OC22 dataset by running:

       cd ocp/
       sh experimental/scripts/train/oc22/s2ef/equiformer_v2/equiformer_dens_v2_N@18_L@6_M@2_epochs@6_g@multi-nodes.sh
   

   This script will use 4 nodes with 8 GPUs on each node.

MD17

Please refer to this repository for training Equiformer with DeNS on MD17.

Checkpoint

We provide the checkpoints of EquiformerV2 trained with DeNS on OC20 S2EF-2M dataset for 12 and 30 epochs, OC20 S2EF-All+MD dataset, and OC22 dataset. |Split |Epochs |Download |val force MAE (meV / Å) |val energy MAE (meV) | |--- |--- |--- |--- |--- | | OC20 S2EF-2M | 12 |checkpoint | config | 19.09 | 269 | | OC20 S2EF-2M | 30 |checkpoint | config | 18.02 | 251 | | OC20 S2EF-All+MD | 2 | checkpoint | config | 14.0 | 222 | | OC22 | 6 | checkpoint | config | (ID) 20.66 | (OOD) 27.11 | (ID) 391.6 | (OOD) 533.0 |

Evaluation

We provide the evaluation script on OC20 and OC22 datasets. After following the above Environment section and