SyntheticTumors
[CVPR 2023] Label-Free Liver Tumor Segmentation
Install / Use
/learn @MrGiovanni/SyntheticTumorsREADME
This repository provides extensive examples of synthetic liver tumors generated by our novel strategies. Check to see if you could tell which is real tumor and which is synthetic tumor. More importantly, our synthetic tumors can be used for training AI models, and have proven to achieve a similar (actually, better) performance in real tumor segmentation than a model trained on real tumors.
Amazing, right?
<p align="center"><img width="100%" src="figures/VisualTuringTest.png" /></p> <p align="center"><img width="100%" src="figures/Examples.gif" /></p>Paper
<b>Label-Free Liver Tumor Segmentation</b> <br/> Qixin Hu<sup>1</sup>, Yixiong Chen<sup>2</sup>, Junfei Xiao<sup>3</sup>, Shuwen Sun<sup>4</sup>, Jieneng Chen<sup>3</sup>, Alan L. Yuille<sup>3</sup>, and Zongwei Zhou<sup>3,*</sup> <br/> <sup>1 </sup>Huazhong University of Science and Technology, <br/> <sup>2 </sup>The Chinese University of Hong Kong -- Shenzhen, <br/> <sup>3 </sup>Johns Hopkins University, <br/> <sup>4 </sup>The First Affiliated Hospital of Nanjing Medical University <br/> CVPR, 2023 <br/> paper | code | talk (by Alan Yuille) | talk | slides | poster
<b>Synthetic Tumors Make AI Segment Tumors Better</b> <br/> Qixin Hu<sup>1</sup>, Junfei Xiao<sup>2</sup>, Yixiong Chen<sup>3</sup>, Shuwen Sun<sup>4</sup>, Jieneng Chen<sup>2</sup>, Alan L. Yuille<sup>2</sup>, and Zongwei Zhou<sup>2,*</sup> <br/> <sup>1 </sup>Huazhong University of Science and Technology, <br/> <sup>2 </sup>Johns Hopkins University, <br/> <sup>3 </sup>The Chinese University of Hong Kong -- Shenzhen, <br/> <sup>4 </sup>The First Affiliated Hospital of Nanjing Medical University <br/> Medical Imaging Meets NeurIPS, 2022 <br/> paper | code | slides | poster | demo | talk
<b>Early Detection and Localization of Pancreatic Cancer by Label-Free Tumor Synthesis</b> <br/> Bowen Li<sup>1</sup>, Yu-Cheng Chou<sup>1</sup>, Shuwen Sun<sup>2</sup>, Hualin Qiao<sup>3</sup>, Alan L. Yuille<sup>1</sup>, and Zongwei Zhou<sup>1,*</sup> <br/> <sup>1 </sup>Johns Hopkins University, <br/> <sup>2 </sup>The First Affiliated Hospital of Nanjing Medical University, <br/> <sup>3 </sup>Sepax technologies <br/> Big Task Small Data, 1001-AI, MICCAI Workshop, 2023 <br/> paper | code
We have documented common questions for the paper in Frequently Asked Questions (FAQ).
We have also provided a list of publications related to tumor synthesis in Awesome Synthetic Tumors 
.
Model
| Tumor | Model | Pre-trained? | Download | | ---- | ---- | ---- | ---- | | real | unet | no | link | | real | swin_unetrv2_base | yes | link | | real | swin_unetrv2_base | no | link | | real | swin_unetrv2_small | no | link | | real | swin_unetrv2_tiny | no | link | | synt | unet | no | link | | synt | swin_unetrv2_base | yes | link | | synt | swin_unetrv2_base | no | link | | synt | swin_unetrv2_small | no | link | | synt | swin_unetrv2_tiny | no | link |
Use the following command to download everything.
wget https://www.dropbox.com/s/jys4tt2ttmr7ig1/runs.tar.gz
tar -xzvf runs.tar.gz
0. Installation
git clone https://github.com/MrGiovanni/SyntheticTumors.git
cd SyntheticTumors
wget https://github.com/Project-MONAI/MONAI-extra-test-data/releases/download/0.8.1/model_swinvit.pt
See installation instructions.
1. Train segmentation models using synthetic tumors
datapath=/mnt/zzhou82/PublicAbdominalData/
# UNET (no.pretrain)
CUDA_VISIBLE_DEVICES=0 python -W ignore main.py --optim_lr=4e-4 --batch_size=2 --lrschedule=warmup_cosine --optim_name=adamw --model_name=unet --val_every=200 --max_epochs=4000 --save_checkpoint --workers=2 --noamp --distributed --dist-url=tcp://127.0.0.1:12235 --cache_num=200 --val_overlap=0.5 --syn --logdir="runs/synt.no_pretrain.unet" --train_dir $datapath --val_dir $datapath --json_dir datafolds/healthy.json
# Swin-UNETR-Base (pretrain)
CUDA_VISIBLE_DEVICES=0 python -W ignore main.py --optim_lr=4e-4 --batch_size=2 --lrschedule=warmup_cosine --optim_name=adamw --model_name=swin_unetrv2 --swin_type=base --val_every=200 --max_epochs=4000 --save_checkpoint --workers=2 --noamp --distributed --dist-url=tcp://127.0.0.1:12231 --cache_num=200 --val_overlap=0.5 --syn --logdir="runs/synt.pretrain.swin_unetrv2_base" --train_dir $datapath --val_dir $datapath --json_dir datafolds/healthy.json --use_pretrained
# Swin-UNETR-Base (no.pretrain)
CUDA_VISIBLE_DEVICES=0 python -W ignore main.py --optim_lr=4e-4 --batch_size=2 --lrschedule=warmup_cosine --optim_name=adamw --model_name=swin_unetrv2 --swin_type=base --val_every=200 --max_epochs=4000 --save_checkpoint --workers=2 --noamp --distributed --dist-url=tcp://127.0.0.1:12231 --cache_num=200 --val_overlap=0.5 --syn --logdir="runs/synt.no_pretrain.swin_unetrv2_base" --train_dir $datapath --val_dir $datapath --json_dir datafolds/healthy.json
# Swin-UNETR-Small (no.pretrain)
CUDA_VISIBLE_DEVICES=0 python -W ignore main.py --optim_lr=4e-4 --batch_size=2 --lrschedule=warmup_cosine --optim_name=adamw --model_name=swin_unetrv2 --swin_type=small --val_every=200 --max_epochs=4000 --save_checkpoint --workers=2 --noamp --distributed --dist-url=tcp://127.0.0.1:12233 --cache_num=200 --val_overlap=0.5 --syn --logdir="runs/synt.no_pretrain.swin_unetrv2_small" --train_dir $datapath --val_dir $datapath --json_dir datafolds/healthy.json
# Swin-UNETR-Tiny (no.pretrain)
CUDA_VISIBLE_DEVICES=0 python -W ignore main.py --optim_lr=4e-4 --batch_size=2 --lrschedule=warmup_cosine --optim_name=adamw --model_name=swin_unetrv2 --swin_type=tiny --val_every=200 --max_epochs=4000 --save_checkpoint --workers=2 --noamp --distributed --dist-url=tcp://127.0.0.1:12234 --cache_num=200 --val_overlap=0.5 --syn --logdir="runs/synt.no_pretrain.swin_unetrv2_tiny" --train_dir $datapath --val_dir $datapath --json_dir datafolds/healthy.json
2. Train segmentation models using real tumors (for comparison)
datapath=/mnt/zzhou82/PublicAbdominalData/
# UNET (no.pretrain)
CUDA_VISIBLE_DEVICES=0 python -W ignore -W ignore main.py --optim_lr=4e-4 --batch_size=2 --lrschedule=warmup_cosine --optim_name=adamw --model_name=unet --val_every=200 --val_overlap=0.5 --max_epochs=4000 --save_checkpoint --workers=2 --noamp --distributed --dist-url=tcp://127.0.0.1:12235 --cache_num=200 --logdir="runs/real.no_pretrain.unet" --train_dir $datapath --val_dir $datapath --json_dir datafolds/lits.json
# Swin-UNETR-Base (pretrain)
CUDA_VISIBLE_DEVICES=0 python -W ignore -W ignore main.py --optim_lr=4e-4 --batch_size=2 --lrschedule=warmup_cosine --optim_name=adamw --model_name=swin_unetrv2 --swin_type=base --val_every=200 --val_overlap=0.5 --max_epochs=4000 --save_checkpoint --workers=2 --noamp --distributed --dist-url=tcp://127.0.0.1:12231 --cache_num=200 --logdir="runs/real.pretrain.swin_unetrv2_base" --train_dir $datapath --val_dir $datapath --json_dir datafolds/lits.json --use_pretrained
# Swin-UNETR-Base (no.pretrain)
CUDA_VISIBLE_DEVICES=0 python -W ignore -W ignore main.py --optim_lr=4e-4 --batch_size=2 --lrschedule=warmup_cosine --optim_name=adamw --model_name=swin_unetrv2 --swin_type=base --val_every=200 --val_overlap=0.5 --max_epochs=4000 --save_checkpoint --workers=2 --noamp --distributed --dist-url=tcp://127.0.0.1:12232 --cache_num=200 --logdir="run
