SCINet

This is the original pytorch implementation for the following paper: SCINet: Time Series Modeling and Forecasting with Sample Convolution and Interaction. Alse see the Open Review verision.

If you find this repository useful for your research work, please consider citing it as follows:

@article{liu2022SCINet,

title={SCINet: Time Series Modeling and Forecasting with Sample Convolution and Interaction},

author={Liu, Minhao and Zeng, Ailing and Chen, Muxi and Xu, Zhijian and Lai, Qiuxia and Ma, Lingna and Xu, Qiang},

journal={Thirty-sixth Conference on Neural Information Processing Systems (NeurIPS), 2022},

year={2022}

}

Updates

• [2022-09-15] SCINet has been accepted to NeurIPS 2022!

• [2021-11-10] Added Reversible Instance Normalization RevIN[1] support!

• [2021-09-17] SCINet v1.0 is released

Features

• [x] Support 11 popular time-series forecasting datasets, namely Electricity Transformer Temperature (ETTh1, ETTh2 and ETTm1) , Traffic, Solar-Energy, Electricity and Exchange Rate and PeMS (PEMS03, PEMS04, PEMS07 and PEMS08), ranging from power, energy, finance and traffic domains.

• [x] Provide all training logs.

• [x] Support RevIN to handle datasets with a large train-test sample distribution gap. To activate, simply add --RIN True to the command line. [Read more](./docs/RevIN.md)

To-do items

• Integrate GNN-based spatial models into SCINet for better performance and higher efficiency on spatial-temporal time series. Our preliminary results show that this feature could result in considerable gains on the prediction accuracy of some datasets (e.g., PEMSxx).

• Generate probalistic forecasting results.

Stay tuned!

Used Datasets

We conduct the experiments on 11 popular time-series datasets, namely Electricity Transformer Temperature (ETTh1, ETTh2 and ETTm1) , PeMS (PEMS03, PEMS04, PEMS07 and PEMS08) and Traffic, Solar-Energy, Electricity and Exchange Rate, ranging from power, energy, finance and traffic domains.

Overall information of the 11 datasets

| Datasets | Variants | Timesteps | Granularity | Start time | Task Type | | ------------- | -------- | --------- | ----------- | ---------- | ----------- | | ETTh1 | 7 | 17,420 | 1hour | 7/1/2016 | Multi-step | | ETTh2 | 7 | 17,420 | 1hour | 7/1/2016 | Multi-step | | ETTm1 | 7 | 69,680 | 15min | 7/1/2016 | Multi-step | | PEMS03 | 358 | 26,209 | 5min | 5/1/2012 | Multi-step | | PEMS04 | 307 | 16,992 | 5min | 7/1/2017 | Multi-step | | PEMS07 | 883 | 28,224 | 5min | 5/1/2017 | Multi-step | | PEMS08 | 170 | 17,856 | 5min | 3/1/2012 | Multi-step | | Traffic | 862 | 17,544 | 1hour | 1/1/2015 | Single-step | | Solar-Energy | 137 | 52,560 | 1hour | 1/1/2006 | Single-step | | Electricity | 321 | 26,304 | 1hour | 1/1/2012 | Single-step | | Exchange-Rate | 8 | 7,588 | 1hour | 1/1/1990 | Single-step |

Get started

Requirements

Install the required package first:

cd SCINet
conda create -n scinet python=3.8
conda activate scinet
pip install -r requirements.txt

Dataset preparation

All datasets can be downloaded here. To prepare all dataset at one time, you can just run:

source prepare_data.sh

The data directory structure is shown as follows.

./
└── datasets/
    ├── ETT-data
    │   ├── ETTh1.csv
    │   ├── ETTh2.csv
    │   └── ETTm1.csv
    ├── financial
    │   ├── electricity.txt
    │   ├── exchange_rate.txt
    │   ├── solar_AL.txt
    │   └── traffic.txt
    └── PEMS
        ├── PEMS03.npz
        ├── PEMS04.npz
        ├── PEMS07.npz
        └── PEMS08.npz

Run training code

We follow the same settings of StemGNN for PEMS 03, 04, 07, 08 datasets, MTGNN for Solar, electricity, traffic, financial datasets, Informer for ETTH1, ETTH2, ETTM1 datasets. The detailed training commands are given as follows.

For PEMS dataset (All datasets follow Input 12, Output 12):

pems03

python run_pems.py --dataset PEMS03 --hidden-size 0.0625 --dropout 0.25 --model_name pems03_h0.0625_dp0.25 --num_decoder_layer 2

pems04

python run_pems.py --dataset PEMS04 --hidden-size 0.0625 --dropout 0 --model_name pems04_h0.0625_dp0

pems07

python run_pems.py --dataset PEMS07 --hidden-size 0.03125 --dropout 0.25 --model_name pems07_h0.03125_dp0.25

pems08

python run_pems.py --dataset PEMS08 --hidden-size 1 --dropout 0.5 --model_name pems08_h1_dp0.5

PEMS Parameter highlights

| Parameter Name | Description | Parameter in paper | Default | | -------------- | ----------------------- | ------------------ | ------- | | dataset | Name of dataset | N/A | PEMS08 | | horizon | Horizon | Horizon | 12 | | window_size | Look-back window | Look-back window | 12 | | hidden-size | hidden expansion | h | 1 | | levels | SCINet block levels | L | 2 | | stacks | The number of SCINet block| K | 1 |

For Solar dataset:

predict 3

python run_financial.py --dataset_name solar_AL --window_size 160 --horizon 3 --hidden-size 1  --lastWeight 0.5 --stacks 2 --levels 4 --lradj 2 --lr 1e-4 --dropout 0.25 --batch_size 256 --model_name so_I160_o3_lr1e-4_bs256_dp0.25_h1_s2l4_w0.5

predict 6

python run_financial.py --dataset_name solar_AL --window_size 160 --horizon 6 --hidden-size 0.5 --lastWeight 0.5 --stacks 2 --levels 4 --lradj 2 --lr 1e-4 --dropout 0.25 --batch_size 256 --model_name so_I160_o6_lr1e-4_bs256_dp0.25_h0.5_s2l4_w0.5 

predict 12

python run_financial.py --dataset_name solar_AL --window_size 160 --horizon 12 --hidden-size 2 --lastWeight 0.5 --stacks 2 --levels 4 --lradj 2 --lr 1e-4 --dropout 0.25 --batch_size 1024 --model_name so_I160_o12_lr1e-4_bs1024_dp0.25_h2_s2l4_w0.5

predict 24

python run_financial.py --dataset_name solar_AL --window_size 160 --horizon 24 --hidden-size 1 --lastWeight 0.5 --stacks 1 --levels 4 --lradj 2 --lr 1e-4 --dropout 0.25 --batch_size 256 --model_name so_I160_o24_lr1e-4_bs256_dp0.25_h1_s1l4_w0.5

For Electricity dataset:

predict 3

python run_financial.py --dataset_name electricity --window_size 168 --horizon 3 --hidden-size 8 --single_step 1 --stacks 2 --levels 3 --lr 9e-3 --dropout 0 --batch_size 32 --model_name ele_I168_o3_lr9e-3_bs32_dp0_h8_s2l3_w0.5 --groups 321 --num_decoder_layer 2

predict 6

python run_financial.py --dataset_name electricity --window_size 168 --horizon 6 --hidden-size 8 --single_step 1 --stacks 2 --levels 3 --lr 9e-3 --dropout 0 --batch_size 32 --model_name ele_I168_o6_lr9e-3_bs32_dp0_h8_s2l3_w0.5 --groups 321 --num_decoder_layer 3

predict 12

python run_financial.py --dataset_name electricity --window_size 168 --horizon 12 --hidden-size 8 --single_step 1 --stacks 2 --levels 3 --lr 9e-3 --dropout 0 --batch_size 32 --model_name ele_I168_o12_lr9e-3_bs32_dp0_h8_s2l3_w0.5 --groups 321 --num_decoder_layer 3

predict 24

python run_financial.py --dataset_name electricity --window_size 168 --horizon 24 --hidden-size 8 --single_step 1 --stacks 2 --levels 3 --lr 9e-3 --dropout 0 --batch_size 32 --model_name ele_I168_o24_lr9e-3_bs32_dp0_h8_s2l3_w0.5 --groups 321 --num_decoder_layer 3

predict 96

python -u run_financial.py --dataset_name electricity --window_size 96 --horizon 96 --hidden-size 8 --stacks 2 --levels 3 --lr 9e-4 --dropout 0 --batch_size 32 --model_name ele_I96_o96_lr9e-4_bs32_dp0_h8_s2l3_w0.5_n4 --groups 321  --concat_len 0   --normalize 4 --long_term_forecast

predict 192

python -u run_financial.py --dataset_name electricity --window_size 96 --horizon 192 --hidden-size 8 --stacks 2 --levels 3 --lr 9e-4 --dropout 0 --batch_size 32 --model_name ele_I96_o192_lr9e-4_bs32_dp0_h8_s2l3_w0.5_n4 --groups 321  --concat_len 0   --normalize 4 --long_term_forecast

predict 336

python -u run_financial.py --dataset_name electricity --window_size 96 --horizon 336 --hidden-size 8 --stacks 2 --levels 3 --lr 9e-4 --dropout 0 --batch_