Image segmentation toolbox

<!-- [](https://app.shippable.com/github/Borda/pyImSegm) [](https://app.shippable.com/github/Borda/pyImSegm) [](https://mybinder.org/v2/gh/Borda/pyImSegm/master?filepath=notebooks) -->

---

Superpixel segmentation with GraphCut regularisation

Image segmentation is widely used as an initial phase of many image processing tasks in computer vision and image analysis. Many recent segmentation methods use superpixels because they reduce the size of the segmentation problem by order of magnitude. Also, features on superpixels are much more robust than features on pixels only. We use spatial regularisation on superpixels to make segmented regions more compact. The segmentation pipeline comprises (i) computation of superpixels; (ii) extraction of descriptors such as colour and texture; (iii) soft classification, using a standard classifier for supervised learning, or the Gaussian Mixture Model for unsupervised learning; (iv) final segmentation using Graph Cut. We use this segmentation pipeline on real-world applications in medical imaging (see sample images. We also show that unsupervised segmentation is sufficient for some situations, and provides similar results to those obtained using trained segmentation.

Sample ipython notebooks:

• Supervised segmentation requires training annotation
• Unsupervised segmentation just asks for expected number of classes
• partially annotated images with missing annotation is marked by a negative number

Illustration

Reference: Borovec J., Svihlik J., Kybic J., Habart D. (2017). Supervised and unsupervised segmentation using superpixels, model estimation, and Graph Cut. In: Journal of Electronic Imaging.

Object centre detection and Ellipse approximation

An image processing pipeline to detect and localize Drosophila egg chambers that consists of the following steps: (i) superpixel-based image segmentation into relevant tissue classes (see above); (ii) detection of egg center candidates using label histograms and ray features; (iii) clustering of center candidates and; (iv) area-based maximum likelihood ellipse model fitting. See our Poster related to this work.

Sample ipython notebooks:

• Center detection consists of center candidate training and prediction, and candidate clustering.
• Ellipse fitting with given estimated center structure segmentation.

Illustration

Reference: Borovec J., Kybic J., Nava R. (2017) Detection and Localization of Drosophila Egg Chambers in Microscopy Images. In: Machine Learning in Medical Imaging.

Superpixel Region Growing with Shape prior

Region growing is a classical image segmentation method based on hierarchical region aggregation using local similarity rules. Our proposed approach differs from standard region growing in three essential aspects. First, it works on the level of superpixels instead of pixels, which leads to a substantial speedup. Second, our method uses learned statistical shape properties which encourage growing leading to plausible shapes. In particular, we use ray features to describe the object boundary. Third, our method can segment multiple objects and ensure that the segmentations do not overlap. The problem is represented as energy minimisation and is solved either greedily, or iteratively using GraphCuts.

Sample ipython notebooks:

• General GraphCut from given centers and initial structure segmentation.
• Shape modeling estimation from training examples.
• Region growing from given centers and initial structure segmentation with shape models.

Illustration

Reference: Borovec J., Kybic J., Sugimoto, A. (2017). Region growing using superpixels with learned shape prior. In: Journal of Electronic Imaging.

---

Installation and configuration

Configure local environment

Create your own local environment, for more see the User Guide, and install dependencies requirements.txt contains list of packages and can be installed as

@duda:~$ cd pyImSegm
@duda:~/pyImSegm$ virtualenv env
@duda:~/pyImSegm$ source env/bin/activate
(env)@duda:~/pyImSegm$ pip install -r requirements.txt
(env)@duda:~/pyImSegm$ python ...

and in the end terminating...

(env)@duda:~/pyImSegm$ deactivate

<!-- Moreover, we are using python [GraphCut wrapper](https://github.com/Borda/pyGCO) which require to be installed separately (not yet integrated in PIP) ```bash (env)@duda:~/pyImSegm$ mkdir libs && cd libs (env)@duda:~/pyImSegm$ git clone https://github.com/Borda/pyGCO.git (env)@duda:~/pyImSegm$ pip install -r requirements.txt (env)@duda:~/pyImSegm$ python setup.py install ``` -->

Compilation

We have implemented cython version of some functions, especially computing descriptors, which require to compile them before using them

python setup.py build_ext --inplace

If loading of compiled descriptors in cython fails, it is automatically swapped to use numpy which gives the same results, but it is significantly slower.

Installation

The package can be installed via pip

pip install git+https://github.com/Borda/pyImSegm.git

or using setuptools from a local folder

python setup.py install

---

Experiments

Short description of our three sets of experiments that together compose single image processing pipeline in this order:

1. Semantic (un/semi)supervised segmentation
2. Center detection and ellipse fitting
3. Region growing with the learned shape prior

Annotation tools

We introduce some useful tools for work with image annotation and segmentation.

• Quantization: in case you have some smooth colour labelling in your images you can remove them with following quantisation script.

  python handling_annotations/run_image_color_quantization.py \
      -imgs "./data-images/drosophila_ovary_slice/segm_rgb/*.png" \
      -m position -thr 0.01 --nb_workers 2
  

• Paint labels: concerting image labels into colour space and other way around.

  python handling_annotations/run_image_convert_label_color.py \
      -imgs "./data-images/drosophila_ovary_slice/segm/*.png" \
      -out ./data-images/drosophila_ovary_slice/segm_rgb
  

• Visualisation: having input image and its segmentation we can use simple visualisation which overlap the segmentation over input image.

  python handling_annotations/run_overlap_images_segms.py \
      -imgs "./data-images/drosophila_ovary_slice/image/*.jpg" \
      -segs ./data-images/drosophila_ovary_slice/segm \
      -out ./results/overlap_ovary_segment
  

• In-painting selected labels in segmentation.

  python handling_annotations/run_segm_annot_inpaint.py \
      -imgs "./data-images/drosophila_ovary_slice/segm/*.png" \
      --label 4
  

• Replace labels: change labels in input segmentation into another set of labels in 1:1 schema.

  python handling_annotations/run_segm_annot_relabel.py \
      -out ./results/relabel_center_levels \
      --label