nd2tool v0.1.8

Introduction

Provides the command line tool, nd2tool, that can be used to:

1. make a short summary of the meta data in a nd2 file (--info),
2. extract all metadata (--meta) from nd2 file or specific portions (--meta-file, --meta-coord, --meta-frame, --meta-text, --meta-exp), and,
3. export the image data to tif file(s). Either one per Field of View (FOV) (--composite), or one file per FOV and color.

The RAM usage is low. Only one image plane is loaded into RAM at the same time. As an example, a 23 GB ND2 image can typically be converted to tiff files using less than 100 Mb of RAM.

nd2tool should be considered as experimental since it is only tested on a few images. If it does not work for your images, please submit a bug report, or simply find a tool that suits you better; some alternatives are listed in the references. At the moment it only supports loops over XY, Color and Z, i.e., not over time. It is furthermore limited to nd2 files where the image data is stored as 16-bit unsigned int.

Supported platforms: Linux. If you want to have this running under macOS or Windows, let me know.

Usage

See the man page for the full documentation (i.e. man nd2tool) or use nd2tool --help for a quick recap.

Basic usage -- conversion

This will convert an nd2 file to a set of tif files. The output will be a folder named after the nd2 file, but without the .nd2 extension.

$ nd2tool iiQV015_20220630_001.nd2
3 FOV in 4 channels:
   #1 '  A647', λ_em=710.0 #E10000  ir
   #2 'SpGold', λ_em=572.0 #FFFF00
   #3 '  A488', λ_em=543.0 #00FF00
   #4 '  dapi', λ_em=385.0 #8900FF  uv
Bits per pixel: 16, significant: 16
dx=324.5 nm, dy=324.5 nm, dz=300.0 nm
NA=0.750, ni=1.000
Objective Name: Plan Apo VC 20x DIC N2
Objective Magnification: 20.0X
Volume size: 2048 x 2048 x 241
Looping: Dimensions: XY(3) x λ(4) x Z(241)
Writing to iiQV015_20220630_001/A647_001.tif
Writing to iiQV015_20220630_001/SpGold_001.tif
...
Writing to iiQV015_20220630_001/dapi_003.tif

Export the coordinates from multi-FOV images

Coordinates are useful for stitching, detection of overlapping regions, detection of mechanical instabilities etc. The example command below gives output in a comma separated format (CSV).

$ nd2tool --coord iMS441_20191016_001.nd2
FOV, Channel, Z, X_um, Y_um, Z_um
1, 1, 1, -3523.100000, 4802.500000, 2120.857777
1, 1, 2, -3523.100000, 4802.500000, 2121.457777
1, 1, 3, -3523.100000, 4802.500000, 2122.057777
...

Generation of deconvolution scripts

nd2tool can generate scripts that deconvolve all images in the nd2 file with deconwolf. Either it creates a script that has to be edited manually --deconwolf or it asks for the key parameters interactively with --deconwolfx as in the example below:

$ nd2tool --deconwolfx iMS441_20191016_001.nd2
Writing to deconwolf_iMS441_20191016_001.nd2.sh
Enter the number of iterations to use
dapi (default=50)
> 50
A647 (default=50)
> 75
Enter any extra arguments to deconwolf
> --gpu --tilesize 1024

For this file the generated file was:

cat deconwolf_iMS441_20191016_001.nd2.sh
#!/bin/env bash
set -e # abort on errors

# PSF Generation
dw_bw --lambda 385.000000 --resxy 108.333333 --resz 600.000000 --NA 1.400000 --ni 1.515000 'iMS441_20191016_001/PSF_dapi.tif'
dw_bw --lambda 710.000000 --resxy 108.333333 --resz 600.000000 --NA 1.400000 --ni 1.515000 'iMS441_20191016_001/PSF_A647.tif'
iter_dapi=50
iter_A647=75
xargs="--gpu --tilesize 1024"
dw "$xargs" --iter $iter_dapi 'iMS441_20191016_001/dapi_001.tif' 'iMS441_20191016_001/PSF_dapi.tif'
dw "$xargs" --iter $iter_A647 'iMS441_20191016_001/A647_001.tif' 'iMS441_20191016_001/PSF_A647.tif'

[!WARNING] The emission wave lengths are typically wrong and might need manual curation.

Generation of spot detection scripts

nd2tool can generate scripts for spot detection that will use deconwolf

Example usage:

$ nd2tool --deconwolf_dots iRB0156_20200217_002.nd2
Writing to deconwolf_dots_iRB0156_20200217_002.nd2.sh
Please set xargs and prefix before running the script

As the note from the program says, please check the script before running it.

[!WARNING] The emission wave lengths are typically wrong and might need manual curation.

Extraction of image data from truncated files

If the microscope crashes while imaging or a file is truncated due to a failed transfer it is sometimes possible to recover some or all of the image data. This tool has only been used once, to recover images from a 87 GB nd2 where NIS elements failed.

<details><summary>tlambert03/nd2 also fails</summary>

>>> import nd2 # pip install nd2
>>> nd2.version('nd2')
'0.11.2'
>>> with nd2.ND2File('iiXZ1299_20260310_002.nd2') as ndfile: print(ndfile.metadata)
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.12/functools.py", line 995, in __get__
    val = self.func(instance)
          ^^^^^^^^^^^^^^^^^^^
  File "/home/elgw/.venv/lib/python3.12/site-packages/nd2/_nd2file.py", line 553, in metadata
    return self._rdr.metadata()
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/elgw/.venv/lib/python3.12/site-packages/nd2/_readers/_modern/modern_reader.py", line 192, in metadata
    raw_meta=self._cached_raw_metadata(),
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/elgw/.venv/lib/python3.12/site-packages/nd2/_readers/_modern/modern_reader.py", line 170, in _cached_raw_metadata
    meta = self._decode_chunk(k, strip_prefix=False)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/elgw/.venv/lib/python3.12/site-packages/nd2/_readers/_modern/modern_reader.py", line 155, in _decode_chunk
    data = self._load_chunk(name)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/elgw/.venv/lib/python3.12/site-packages/nd2/_readers/_modern/modern_reader.py", line 130, in _load_chunk
    offset = self.chunkmap[name][0]
             ^^^^^^^^^^^^^
  File "/home/elgw/.venv/lib/python3.12/site-packages/nd2/_readers/_modern/modern_reader.py", line 102, in chunkmap
    self._chunkmap = get_chunkmap(self._fh, error_radius=self._error_radius)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/elgw/.venv/lib/python3.12/site-packages/nd2/_parse/_chunk_decode.py", line 152, in get_chunkmap
    raise ValueError(f"Invalid ChunkMap signature {sig!r} in file {fh.name!r}")
ValueError: Invalid ChunkMap signature b'\xa1\x04\xc6\x07Q\x03e\x00\xcc\x01\x0e\x02\xab\x04\x83\x07Z\x03s\x00:\x026\x02{\x04\xeb\x07\xf2\x03k\x00' in file '/mnt/TOSH_18T/FFPE_FISH/iiXZ1299_20260310_002.nd2'

</details>

The recovery process has two steps:

1. Find the location of the image data with the nd2tool_recover binary.
2. Use the locations to extract the images with recover_from_locations.py from the scripts folder in this repo.

No meta-data is read from the file and hence you will have to provide the image sizes, the bit/byte depth and the number of channels.

It assumes that all images have the same size, etc, so it might not be a general solution. If you have a better option I'd be happy to link to it. The channels will be named c01, c02, etc.

Example usage:

# Find position of images in the nd2 file
$ nd2tool_recover iiXZ1299_20260310_002.nd2
iiXZ1299_20260310_002.nd2 is 93021286400 b
Will write to iiXZ1299_20260310_002.nd2.recover_locations.csv
.... ...@.......ND2 FILE SIGNATURE CHUNK NAME01!Ver3.0......
Found 1848 ImageDataSeq
Each ImageDataSeq can have at most 50331664 bytes
# Extract images
$ python scripts/recover_from_locations.py --nd2 iiXZ1299_20260310_002.nd2 --width 2048 --height 2048 --planes 77 --ncolor 6
Expected chunk size: 50331648 (Width x Height x nColor x BPS)
Parameters are feasible, chunks are at least 50331664 bytes
Creating iiXZ1299_20260310_002_recover/
Will try to recover 24 FOV
Writing iiXZ1299_20260310_002_recover/c01_001.tif
Writing iiXZ1299_20260310_002_recover/c02_001.tif
Writing iiXZ1299_20260310_002_recover/c03_001.tif
...
# When you are done and know which channel is which you could continue with command like
$ file-rename --nono -- 's/c01/dapi/' c*.tif # remove --nono to do the job

Installation

The simplest way to use nd2tool is to download the AppImage, see the releases

The standard procedure: get required libraries, compile and then install. These instructions are for Ubuntu 22.04 LTS (might work on WSL for Windows as well).

get dependencies

sudo apt-get update
sudo apt-get install libcjson1 libcjson-dev libtiff5-dev build-essential

compile

mkdir build
cd build
cmake ..
cmake build

install

Either bypass the system package manager

make install

or do it the proper way

make package
sudo apt-get install ./nd2tool-0.1.7-Linux.deb

of course, you will need to adjust the last line to fit the system package manager unless you are on Ubuntu.

nd2tool_recover is not built by default, to build it, use:

mkdir build
cd build
cmake ..
make nd2tool_recover

Reporting bugs

Please use the issue tracking system on github to report bugs or to get in touch. Have a look on the roadmap before submitting suggestions or making pull requests.

References

Alternative command line tools for nd2 files:

• ggirelli/radiantkit - Command line tool for conversion from nd2 to tiff as well as lots of other stuff. Unfortunately the repository has been archived but the code is still useful. Uses the Python package nd2reader.

GUI tools for nd2 files:

• [Nikon NIS Elements (Viewer)](https://www.microscope.healthcare.nikon.com/products/software/nis-elements/view