Addressing Spatial-Temporal Heterogeneity: General Mixed Time Series Analysis via Latent Continuity Recovery and Alignment

This is the official implementation of MiTSformer in NeurIPS 2024.

</div> <div align="center">

</div> <p align="center"> 🌟 <a href="#-Mixed-Time-Series">Mixed Time Series</a> | 🚀 <a href="#-MiTSformer">MiTSformer</a> | ✅ <a href="#-Setup">Setup</a> | 📊 <a href="#-Train-and-Evaluation">Train and Evaluation</a> </p>

MiTSformer is a task-general mixed time series analysis framework, which recovers and aligns latent continuity of mixed variables for complete and reliable spatial-temporal modeling, being amenable to various analysis tasks and establishing SOTA performance.

🌟 Mixed Time Series

Mixed time series, encompassing both continuous variables and discrete variables are frequently encountered in practice. Mixed time series presents the spatial-temporal heterogeneity problem, which is caused by the discrepancies in temporal variation properties and distribution types between continuous variables and discrete variables:

Mixed time series and spatial-temporal heterogeneity problem

🚀 MiTSformer

MiTSformer addresses the spatial-temporal heterogeneity problem from the perspective of latent continuity recovery and alignment. The architecture and its key components of MiTSformer are shown below:

MiTSformer

✅ Setup

1. Create conda environment(Optional)

conda create -n mitsformer -y python=3.9.7 
conda activate mitsformer

2. Install dependencies

Install the required packages

pip install -r requirements.txt

3. Prepare dataset

All the datasets used in our paper are publicly available. We provide the relevant references and links in the article. Also, you can obtain the well pre-processed datasets from Google Drive or Baidu Drive

|Task|Dataset|Reference/Link| |:-|:-|:-| |Classification|UEA Archive (10 datasets)|Link| |Extrinsic Regression|TSER Repository (10 datasets)|Link| |Anomaly Detection|SMD|Paper| ||MSL|Paper| ||SMAP|Paper| ||SWaT|Paper| ||PSM|Paper| |Imputation|ETT(4 subsets)|Paper| ||Weather|Link| ||Electricity|Link| |Long-term Forecasting|ETT(4 subsets)|Paper| ||Weather|Link| ||Electricity|Link| ||Exchange|Paper| ||ILI|Link|

Then place the downloaded data in the corresponding folder as

./dataset/classification
./dataset/extrinsic_regression
./dataset/long_term_forecast
./dataset/imputation
./dataset/anomaly_detection

Here is a summary of supported datasets. Summary of experiment benchmarks

For each dataset, we randomly select (n = ⌊0.5p⌋) variables as DVs, whose values are first Min-max normalized and then discretized into the value of 0 or 1 with the threshold 0.5 as int(MinMax-Norm(x) > 0.5).

📊 Train and Evaluation.

We provide the experiment scripts for MiTSformer under the folder ./scripts/ e.g., for mixed time series classification task:

mkdir -p ./result_run_classification_logs
python -u run_classification.py --data_name 'EthanolConcentration'  |tee ./result_run_classification_logs/thanolConcentration.log

for mixed time series extrinsic regression task:

mkdir -p ./result_ext_reg_logs
python -u run_extrinsic_regression.py --data_name 'HouseholdPowerConsumption1' |tee ./result_ext_reg_logs/house1.log

for mixed time series imputation task:

mkdir -p ./result_imputation_logs
python -u run_imputation.py --data_name 'ETTh1'  --mask_rate 0.125 |tee ./result_imputation_logs/ETTh1_125.log

for mixed time series anomaly detection task:

mkdir -p ./result_ad_logs
python -u run_anomaly_detection.py --data_name 'SMD' --data 'SMD'   |tee ./result_ad_logs/SMD.log

for mixed time series long-term forecasting task:

mkdir -p ./result_long_term_forecast_logs
python -u run_long_term_forecasting.py --data_name 'ETTh1' --pred_len 96 |tee ./result_long_term_forecast_logs/ETTh1_96.log

Also, you can specify GPU and tune other hyperparameters, e.g., adding:

--gpu 0 --d_model 128 --n_layers 1 --smooth_loss_w 0.5 --learning_rate 0.001

🔗 Citation

If you find our work useful, please consider citing our paper:

Addressing Spatial-Temporal Heterogeneity: General Mixed Time Series Analysis via Latent Continuity Recovery and Alignment. Jiawei Chen and Chunhui Zhao, In the Thirty-Eighth Annual Conference on Neural Information Processing Systems (NeurIPS), 2024, 37: 17910-17946.
Addressing Information Asymmetry: Deep Temporal Causality Discovery for Mixed Time Series. Jiawei Chen and Chunhui Zhao, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 47, no. 7, pp. 5723-5741, July 2025, doi: 10.1109/TPAMI.2025.3553957.

@article{chen2024addressing,
  title={Addressing spatial-temporal heterogeneity: General mixed time series analysis via latent continuity recovery and alignment},
  author={Chen, Jiawei},
  journal={Advances in Neural Information Processing Systems},
  volume={37},
  pages={17910--17946},
  year={2024}
}

@ARTICLE{10937933,
  author={Chen, Jiawei and Zhao, Chunhui},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence}, 
  title={Addressing Information Asymmetry: Deep Temporal Causality Discovery for Mixed Time Series}, 
  year={2025},
  volume={47},
  number={7},
  pages={5723-5741},
  doi={10.1109/TPAMI.2025.3553957}}

Acknowledgement

We appreciate the following github repos a lot for their valuable code base or datasets:

https://github.com/thuml/Time-Series-Library

https://github.com/zhouhaoyi/Informer2020

https://github.com/thuml/Anomaly-Transformer

https://github.com/thuml/Flowformer