SuPreM
[ICLR 2024 Oral] Supervised Pre-Trained 3D Models for Medical Image Analysis (9,262 CT volumes + 25 annotated classes)
Install / Use
/learn @MrGiovanni/SuPreMREADME
We developed a suite of pre-trained 3D models, named SuPreM, that combined the best of large-scale datasets and per-voxel annotations, showing the transferability across a range of 3D medical imaging tasks.
Paper
<b>AbdomenAtlas: A Large-Scale, Detailed-Annotated, & Multi-Center Dataset for Efficient Transfer Learning and Open Algorithmic Benchmarking</b> <br/> Wenxuan Li, Chongyu Qu, Xiaoxi Chen, Pedro R. A. S. Bassi, Yijia Shi, Yuxiang Lai, Qian Yu, Huimin Xue, Yixiong Chen, Xiaorui Lin, Yutong Tang, Yining Cao, Haoqi Han, Zheyuan Zhang, Jiawei Liu, Tiezheng Zhang, Yujiu Ma, Jincheng Wang, Guang Zhang, Alan Yuille, Zongwei Zhou* <br/> Johns Hopkins University <br/> Medical Image Analysis, 2024 <br/> <a href='https://www.zongweiz.com/dataset'><img src='https://img.shields.io/badge/Project-Page-Green'></a> <a href='https://www.cs.jhu.edu/~zongwei/publication/li2024abdomenatlas.pdf'><img src='https://img.shields.io/badge/Paper-PDF-purple'></a>
<b>How Well Do Supervised 3D Models Transfer to Medical Imaging Tasks?</b> <br/>
Wenxuan Li, Alan Yuille, and Zongwei Zhou<sup>*</sup> <br/>
Johns Hopkins University <br/>
International Conference on Learning Representations (ICLR) 2024 (oral; top 1.2%) <br/>
<a href='https://www.zongweiz.com/dataset'><img src='https://img.shields.io/badge/Project-Page-Green'></a> <a href='https://www.cs.jhu.edu/~zongwei/publication/li2023suprem.pdf'><img src='https://img.shields.io/badge/Paper-PDF-purple'></a> <a href='document/promotion_slides.pdf'><img src='https://img.shields.io/badge/Slides-PDF-orange'></a> <a href='document/dom_wse_poster.pdf'><img src='https://img.shields.io/badge/Poster-PDF-blue'></a> 
<a href='https://www.cs.jhu.edu/news/ai-and-radiologists-unite-to-map-the-abdomen/'><img src='https://img.shields.io/badge/WSE-News-yellow'></a>
<b>Transitioning to Fully-Supervised Pre-Training with Large-Scale Radiology ImageNet for Improved AI Transferability in Three-Dimensional Medical Segmentation</b> <br/> Wenxuan Li<sup>1</sup>, Junfei Xiao<sup>1</sup>, Jie Liu<sup>2</sup>, Yucheng Tang<sup>3</sup>, Alan Yuille<sup>1</sup>, and Zongwei Zhou<sup>1,*</sup> <br/> <sup>1</sup>Johns Hopkins University <br/> <sup>2</sup>City University of Hong Kong <br/> <sup>3</sup>NVIDIA <br/> Radiological Society of North America (RSNA) 2023 <br/> <a href='document/rsna_abstract.pdf'><img src='https://img.shields.io/badge/Abstract-PDF-purple'></a> <a href='document/rsna2023_slides.pdf'><img src='https://img.shields.io/badge/Slides-2023-orange'></a> <a href='document/rsna2024_slides.pdf'><img src='https://img.shields.io/badge/Slides-2024-orange'></a>
★ We have maintained a document for Frequently Asked Questions.
★ We have maintained a paper list for Awesome Medical Segment Anything Model.
★ We have maintained a paper list for Awesome Medical Pre-Training.
★ We have maintained a paper list for Awesome Medical Segmentation Backbones.
An Extensive Dataset: AbdomenAtlas 1.1
The release of AbdomenAtlas 1.0 can be found at https://huggingface.co/datasets/AbdomenAtlas/AbdomenAtlas1.0Mini
AbdomenAtlas 1.1 is an extensive dataset of 9,262 CT volumes with per-voxel annotation of 25 organs and pseudo annotations for seven types of tumors, enabling us to finally perform supervised pre-training of AI models at scale. Based on AbdomenAtlas 1.1, we also provide a suite of pre-trained models comprising several widely recognized AI models.
<p align="center"><img width="100%" src="document/fig_benchmark.png" /></p>Prelimianry benchmark showed that supervised pre-training strikes as a preferred choice in terms of performance and efficiency compared with self-supervised pre-training.
We anticipate that the release of large, annotated datasets (AbdomenAtlas 1.1) and the suite of pre-trained models (SuPreM) will bolster collaborative endeavors in establishing Foundation Datasets and Foundation Models for the broader applications of 3D volumetric medical image analysis.
The AbdomenAtlas 1.1 dataset is organized as
AbdomenAtlas1.1
├── BDMAP_00000001
│ ├── ct.nii.gz
│ └── segmentations
│ ├── aorta.nii.gz
│ ├── gall_bladder.nii.gz
│ ├── kidney_left.nii.gz
│ ├── kidney_right.nii.gz
│ ├── liver.nii.gz
│ ├── pancreas.nii.gz
│ ├── postcava.nii.gz
│ ├── spleen.nii.gz
│ ├── stomach.nii.gz
│ └── ...
├── BDMAP_00000002
│ ├── ct.nii.gz
│ └── segmentations
│ ├── aorta.nii.gz
│ ├── gall_bladder.nii.gz
│ ├── kidney_left.nii.gz
│ ├── kidney_right.nii.gz
│ ├── liver.nii.gz
│ ├── pancreas.nii.gz
│ ├── postcava.nii.gz
│ ├── spleen.nii.gz
│ ├── stomach.nii.gz
│ └── ...
├── BDMAP_00000003
│ ├── ct.nii.gz
│ └── segmentations
│ ├── aorta.nii.gz
│ ├── gall_bladder.nii.gz
│ ├── kidney_left.nii.gz
│ ├── kidney_right.nii.gz
│ ├── liver.nii.gz
│ ├── pancreas.nii.gz
│ ├── postcava.nii.gz
│ ├── spleen.nii.gz
│ ├── stomach.nii.gz
│ └── ...
...
# class map for the AbdomenAtlas 1.0 dataset
class_map_abdomenatlas_1_0 = {
1: 'aorta',
2: 'gall_bladder',
3: 'kidney_left',
4: 'kidney_right',
5: 'liver',
6: 'pancreas',
7: 'postcava',
8: 'spleen',
9: 'stomach',
}
# class map for the AbdomenAtlas 1.1 dataset
class_map_abdomenatlas_1_1 = {
1: 'aorta',
2: 'gall_bladder',
3: 'kidney_left',
4: 'kidney_right',
5: 'liver',
6: 'pancreas',
7: 'postcava',
8: 'spleen',
9: 'stomach',
10: 'adrenal_gland_left',
11: 'adrenal_gland_right',
12: 'bladder',
13: 'celiac_trunk',
14: 'colon',
15: 'duodenum',
16: 'esophagus',
17: 'femur_left',
18: 'femur_right',
19: 'hepatic_vessel',
20: 'intestine',
21: 'lung_left',
22: 'lung_right',
23: 'portal_vein_and_splenic_vein',
24: 'prostate',
25: 'rectum'
}
A Suite of Pre-trained Models: SuPreM
The following is a list of supported model backbones in our collection. Select the appropriate family of backbones and click to expand the table, download a specific backbone and its pre-trained weights (name and download), and save the weights into ./pretrained_weights/. More backbones will be added along time. Please suggest the backbone in this channel if you want us to pre-train it on AbdomenAtlas 1.1 containing 9,262 annotated CT volumes.
| name | params | pre-trained data | resources | download |
|:---- |:---- |:---- |:---- |:---- |
| Tang et al. | 62.19M | 5050 CT | 
| weights |
| Jose Valanaras et al. | 62.19M | 50000 CT/MRI | | weights |
| Universal Model | 62.19M | 2100 CT | 
| weights |
| SuPreM | 62.19M | 2100 CT | ours :star2: | weights |
</
