<div align="center"> <h1>HyperSIGMA: Hyperspectral Intelligence Comprehension Foundation Model</h1> <h2>TPAMI 2025</h2>

Di Wang^{1 ∗}, Meiqi Hu^{1 ∗}, Yao Jin^{1 ∗}, Yuchun Miao^{1 ∗}, Jiaqi Yang^{1 ∗}, Yichu Xu^{1 ∗}, Xiaolei Qin^{1 ∗}, Jiaqi Ma^{1 ∗}, Lingyu Sun^{1 ∗}, Chenxing Li^{1 ∗}, Chuan Fu², Hongruixuan Chen³, Chengxi Han^{1 †}, Naoto Yokoya³, Jing Zhang^{1 †}, Minqiang Xu⁴, Lin Liu⁴, Lefei Zhang¹, Chen Wu^{1 †}, Bo Du^{1 †}, Dacheng Tao⁵, Liangpei Zhang^{1 †}

¹ Wuhan University, ² Chongqing University, ³ The University of Tokyo, ⁴ National Engineering Research Center of Speech and Language Information Processing, ⁵ Nanyang Technological University.

^∗ Equal contribution, ^† Corresponding author

</div> <div align="center"> <p align='center'> <a href="https://whu-sigma.github.io/HyperSIGMA/"><img alt="Project" src="https://img.shields.io/badge/Project-Page-375BD2?style=for-the-badge" /></a> <a href="https://arxiv.org/abs/2406.11519"><img alt="Paper" src="https://img.shields.io/badge/arXiv-2406.11519-92003B?style=for-the-badge" /></a> <a href="https://ieeexplore.ieee.org/document/10949864"><img alt="Pape" src="https://img.shields.io/badge/TPAMI-Paper-6D4AFF?style=for-the-badge" /></a> <a href="#"><img alt="GitHub Repo stars" src="https://img.shields.io/github/stars/WHU-Sigma/HyperSIGMA?style=for-the-badge" /></a> <a href="#"><img alt="Maintenance" src="https://img.shields.io/badge/Maintaining-YES-93b023?&style=for-the-badge" /></a> </p> <p align='center'> <a href="https://huggingface.co/WHU-Sigma/HyperSIGMA/tree/main"><img alt="HuggingFace" src="https://img.shields.io/badge/-HuggingFace-FDEE21?style=for-the-badge&logo=HuggingFace&logoColor=black" /></a> <a href="https://github.com/WHU-Sigma/HyperSIGMA/#-pretrained-models"><img alt="Baidu Drive" src="https://img.shields.io/badge/-Baidu%20Drive-87CEEB?style=for-the-badge&logo=Baidu&logoColor=black" /></a> <a href="https://mp.weixin.qq.com/s/dsUYTbZKfYBGmC4mju0zPg"><img alt="Wechat" src="https://img.shields.io/badge/WeChat%20article-07C160?style=for-the-badge&logo=wechat&logoColor=white" /></a> </p> <!-- [](https://arxiv.org/abs/2406.11519) [](https://mp.weixin.qq.com/s/tYqe95Ip-fRBM57F2F5rvw) [](https://github.com/WHU-Sigma/HyperSIGMA) [](https://huggingface.co/WHU-Sigma/HyperSIGMA/tree/main) --> </div> <p align="center"> <a href="#-update">Update</a> | <a href="#-overview">Overview</a> | <a href="#-datasets">Datasets</a> | <a href="#-pretrained-models">Pretrained Models</a> | <a href="#-usage">Usage</a> | <a href="#-statement">Statement</a> </p > <figure> <div align="center"> <img src=Fig/logo1.png width="20%"> </div> </figure>

🔥 Update

2025.11.24

• HyperGlobal-450K can be assesed through <a href="https://pan.baidu.com/s/1duYGTpeEcuQkLByTSjyOtQ?pwd=j9pv">Baidu Drive (百度网盘) <img height="15" width="15" src="https://cdn.jsdelivr.net/npm/simple-icons@v13/icons/baidu.svg"/></a>.

2025.11.15

• 🏆 HyperSIGMA is selected as a Highly Cited Paper!

2025.04.08

• The main paper is online published! Please see here.

2025.04.02

• The arXiv is updated with more details. Stay tuned for further updates!
• HyperGlobal-450K has been released! Please refer to here.

2025.03.31

• We are delighted to annouce HyperSIGMA has been accepted by IEEE TPAMI!

2024.10.22

• Scripts for Image Super-Resolution.

• Checkpoints for Image Denoising.

2024.07.18

• Models can be downloaded from both <a href="#-pretrained-models">Baidu Drive (百度网盘) <img height="15" width="15" src="https://cdn.jsdelivr.net/npm/simple-icons@v13/icons/baidu.svg"/></a> and Hugging Face 🤗.

• Datasets for HSI denoising have been released for research use only. Please check it here.

2024.06.18

• The paper is post on arXiv! (arXiv 2406.11519)

🌞 Overview

HyperSIGMA is the first billion-level foundation model specifically designed for HSI interpretation. To tackle the spectral and spatial redundancy challenges in HSIs, we introduce a novel sparse sampling attention (SSA) mechanism, which effectively promotes the learning of diverse contextual features and serves as the basic block of HyperSIGMA. HyperSIGMA integrates spatial and spectral features using a specially designed spectral enhancement module.</a>

<figure> <div align="center"> <img src=Fig/framework.png width="80%"> </div> <div align='center'>

Figure 1. Framework of HyperSIGMA.

</div> <br>

Extensive experiments on various high-level and low-level HSI tasks demonstrate HyperSIGMA’s versatility and superior representational capability compared to current state-of-the-art methods. It outperforms advanced models like SpectralGPT, even those specifically designed for these tasks.

<figure> <div align="center"> <img src=Fig/radarimg.png width="80%"> </div> </figure>

Figure 2. HyperSIGMA demonstrates superior performance across 16 datasets and 7 tasks, including both high-level and low-level hyperspectral tasks, as well as multispectral scenes.

📖 Datasets

To train the foundational model, we collected hyperspectral remote sensing image samples from around the globe, constructing a large-scale hyperspectral dataset named HyperGlobal-450K for pre-training. HyperGlobal-450K contains over 20 million three-band images, far exceeding the scale of existing hyperspectral datasets.

<figure> <div align="center"> <img src=Fig/dataset.png width="80%"> </div> </figure>

Figure 3. The distribution of HyperGlobal-450K samples across the globe, comprising 1,701 images (1,486 EO-1 and 215 GF-5B) with hundreds of spectral bands.

🚀 Pretrained Models

| Pretrain | Backbone | Model Weights | | :------- | :------: | :------: | | Spatial_MAE | ViT-B | Baidu Drive & Hugging Face| | Spatial_MAE | ViT-L | Baidu Drive & Hugging Face| | Spatial_MAE | ViT-H | Baidu Drive & Hugging Face| | Spectral_MAE | ViT-B | Baidu Drive & Hugging Face| | Spectral_MAE | ViT-L | Baidu Drive & Hugging Face| | Spectral_MAE | ViT-H | Baidu Drive & Hugging Face |

🔨 Usage

Pretraining

We pretrain the HyperSIGMA with SLURM. This is an example of pretraining the large version of Spatial ViT:

srun -J spatmae -p xahdnormal --gres=dcu:4 --ntasks=64 --ntask