NeSVoR: Neural Slice-to-Volume Reconstruction

NeSVoR is a package for GPU-accelerated slice-to-volume reconstruction (both rigid and deformable).

This package is the accumulation of the following works:

[1] SVoRT: Iterative Transformer for Slice-to-Volume Registration in Fetal Brain MRI (Springer | Arxiv)

[2] NeSVoR: Implicit Neural Representation for Slice-to-Volume Reconstruction in MRI (IEEE | TechRxiv)

Overview
- Methods
- Pipeline
Documentation
Installation
- Docker Image
- From Source
Quick Start
Usage
- Reconstruction
- Registration (Motion Correction)
- SVR
- Sampling
 - Sample Volume
 - Sample Slices
- Preprocessing
 - Brain Masking
 - Bias Field Correction
 - Stack Quality Assessment
Cite Our Work
Resources

Overview

Methods

NeSVoR is a deep learning package for solving slice-to-volume reconstruction problems (i.e., reconstructing a 3D isotropic high-resolution volume from a set of motion-corrupted low-resolution slices) with application to fetal/neonatal MRI, which provides

Motion correction by mapping 2D slices to a 3D canonical space using Slice-to-Volume Registration Transformers (SVoRT).
Volumetric reconstruction of multiple 2D slices using implicit neural representation (NeSVoR).

<img src="./docs/_static/images/SVoRT_network.png" align="center" width="600"> Figure 1. SVoRT: an iterative Transformer for slice-to-volume registration. (a) The k-th iteration of SVoRT. (b) The detailed network architecture of the SVT module.  <img src="./docs/_static/images/NeSVoR.png" align="center" width="900"> Figure 2. NeSVoR: A) The forward imaging model in NeSVoR. B) The architecture of the implicit neural network in NeSVoR.

Pipeline

To make our reconstruction tools more handy, we incorporate several preprocessing and downstream analysis tools in this package. The next figure shows our overall reconstruction pipeline.

<img src="./docs/_static/images/pipeline.png" align="center" width="900"> Figure 3. The reconstruction pipeline.

Documentation

The full documentation is available at Read the Docs.

Installation

Docker Image

We recommend to use our docker image to run nesvor.

Install Docker and NVIDIA Container Toolkit

You may follow this guide to install Docker and NVIDIA Container Toolkit

Download NeSVoR Image

docker pull junshenxu/nesvor

Note: our latest image was built with CUDA 11.7.

Run NeSVoR with Docker

You may run a container in an interactive way.

docker run -it --gpus all --ipc=host junshenxu/nesvor
nesvor -h

You may also run the nesvor command directly as follows.

docker run --rm --gpus all --ipc=host \
    -v <path-to-inputs>:/incoming:ro -v <path-to-outputs>:/outgoing:rw \
    junshenxu/nesvor \
    nesvor reconstruct \
    --input-stacks /incoming/stack-1.nii.gz ... /incoming/stack-N.nii.gz \
    --thicknesses <thick-1> ... <thick-N> \
    --output-volume /outgoing/volume.nii.gz

From Source

Check out our documentation if you want to install NeSVoR from source.

Quick Start

Fetal Brain Reconstruction

This example reconstruct a 3D fetal brain from mutiple stacks of 2D images in the following steps:

Segment fetal brain from each image using a CNN (--segmentation).
Correct bias field in each stack using the N4 algorithm (--bias-field-correction).
Register slices using SVoRT (--registration svort).
Reconstruct a 3D volume using NeSVoR.

nesvor reconstruct \
    --input-stacks stack-1.nii.gz ... stack-N.nii.gz \
    --thicknesses <thick-1> ... <thick-N> \
    --output-volume volume.nii.gz \
    --output-resolution 0.8 \
    --registration svort \
    --segmentation \
    --bias-field-correction

Neonatal Brain Reconstruction

This example reconstruct a 3D neonatal brain from mutiple stacks of 2D images in the following steps:

Removal background (air) with Otsu thresholding (--otsu-thresholding).
Correct bias field in each stack using the N4 algorithm (--bias-field-correction).
Register slices using SVoRT (--registration svort).
Reconstruct a 3D volume using NeSVoR.

nesvor reconstruct \
    --input-stacks stack-1.nii.gz ... stack-N.nii.gz \
    --thicknesses <thick-1> ... <thick-N> \
    --output-volume volume.nii.gz \
    --output-resolution 0.8 \
    --registration svort \
    --otsu-thresholding \
    --bias-field-correction

Fetal Body/Uterus Reconstruction

This is an example for deformable NeSVoR which consists of the following steps:

Create an ROI based on the intersection of all input stacks (--stacks-intersection).
Perform stack-to-stack registration (--registration stack).
Reconstruct a 3D volume using Deformable NeSVoR (--deformable).

nesvor reconstruct \
    --input-stacks stack-1.nii.gz ... stack-N.nii.gz \
    --thicknesses <thick-1> ... <thick-N> \
    --output-volume volume.nii.gz \
    --output-resolution 1.0 \
    --stacks-intersection \
    --registration stack \
    --deformable \
    --weight-transformation 1 \
    --weight-deform 0.1 \
    --weight-image 0.1 \
    --single-precision \
    --log2-hashmap-size 22 \
    --batch-size 8192

Usage

This section provides the basic usage of the commands in NeSVoR. Please refer to our document for details. NeSVoR currently supports the following commands:

nesvor reconstruct: reconstruct a 3D volume (i.e., train a NeSVoR model) from either multiple stacks of slices (NIfTI) or a set of motion-corrected slices (the output of register). It can also perform multiple preprocessing steps, including brain segmentation, bias field correction, and registration.
nesvor register: register stacks of slices using a pretrained SVoRT model or stack-to-stack registration.
svr: a classical slice-to-volume registration/reconstruciton method.
nesvor sample-volume: sample a volume from a trained NeSVoR model.
nesvor sample-slices: simulate slices from a trained NeSVoR model.
nesvor segment-stack: 2D fetal brain segmentation/masking in input stacks.
nesvor correct-bias-field: bias field correction using the N4 algorihtm.
nesvor assess: quality and motion assessment of input stacks.

run nesvor <command> -h for a full list of parameters of a command.

Reconstruction

Reconstruct from Mutiple Stacks of Slices

The reconstruct command can be used to reconstruct a 3D volume from N stacks of 2D slices (in NIfTI format, i.e. .nii or .nii.gz). A basic usage of reconstruct is as follows, where mask-i.nii.gz is the ROI mask of the i-th input stack.

nesvor reconstruct \
    --input-stacks stack-1.nii.gz ... stack-N.nii.gz \
    --stack-masks mask-1.nii.gz ... mask-N.nii.gz \
    --output-volume volume.nii.gz

A more elaborate example could be

nesvor reconstruct \
    --input-stacks stack-1.nii.gz ... stack-N.nii.gz \
    --stack-masks mask-1.nii.gz ... mask-N.nii.gz \
    --thicknesses <thick-1> ... <thick-N> \
    --output-volume volume.nii.gz \
    --output-resolution 0.8 \
    --output-model model.pt \
    --weight-image 1.0 \
    --image-regularization edge \
    --delta 0.2 \
    --n-iter 5000 \
    --level-scale 1.38 \
    --coarsest-resolution 16.0 \
    --finest-resolution 0.5 \
    --n-levels-bias 0

NeSVoR

Install / Use

README