JDLL (Java Deep Learning Library)

This project provides a Java library for running Deep Learning (DL) models agnostically, enabling communication between Java software and various Deep Learning frameworks (engines). It also allows the use of multiple DL frameworks in the same session, manages the different DL frameworks and brings the models from the Bioimage.io repository to Java.

JDLL was originally developed by the Icy team as the foundational component for the DeepIcy plugin; however, it evolved over time into a standalone library designed to facilitate the integration of Deep Learning methodologies into other software applications. Its first target was software developers but the Icy team realised that other potential end-users, such as Bioimage analysts with some Python background, could also benefit from it using JDLL for scripting with Jython. Some example scripts can be found here. Also have a look at the section Quickstart for end-users to get more information about how to use JDLL for scripting.

JDLL is able to load models and make inference, create tensors, download Bioimage.io models and manage the supported DL frameworks. The library is designed in a modular way, allowing the main software to avoid dealing with the various objects and structures required by different DL frameworks. Instead the Java model runner provides interfaces for models and tensors that handle internally their creation and inference in the differnet Java engines. The main software only needs to interact with the Java model runner and does not need to worry whether the model is in PyTorch, Tensorflow or other framework.

FOR A MORE COMPREHENSIVE AND COMPLETE EXPLANATION OF JDLL, PLEASE VISIT THE WIKI.

Table of Contents

1. Supported engines
2. Quickstart
3. Examples for developers
4. Scripting examples
5. Quickstart for developers
6. Quickstart for analysts/scripting
7. Acknowledgements
8. References

Supported engines

Currently, the following frameworks are supported:

| Framework | Source code | Tag used in JDLL to refer to the framework | |---------------------------------|----------------------------------------------------------------|------------------------------------------------| | PyTorch | https://github.com/bioimage-io/pytorch-java-interface | pytorch or torchscript | | Tensorflow 1 | https://github.com/bioimage-io/tensorflow-1-java-interface | tensorflow or tensorflow_saved_model_bundle | | Tensorflow 2 API 0.2.0 | https://github.com/bioimage-io/tensorflow-2-java-interface-0.2.0 | tensorflow or tensorflow_saved_model_bundle | | Tensorflow 2 API 0.3-0.4 | https://github.com/bioimage-io/tensorflow-2-java-interface | tensorflow or tensorflow_saved_model_bundle | | Tensorflow 2 API 0.5.0 | https://github.com/bioimage-io/tensorflow-2-java-interface-0.5.0 | tensorflow or tensorflow_saved_model_bundle | | Onnx | https://github.com/bioimage-io/onnx-java-interface | onnx |

The information about the engines supported currently by JDLL, for which OS and architectures, their minimum required Java version and which JAR files are required for each of the engines is stored in this json file and can be found here.

Note that JDLL will be in constant development and that it is open to community collaboration, so pull requests to the official repository of JDLL to improve functionality or to add new engines are very welcomed.

Quickstart

As mentioned above, JDLL was originally developed for the purpose of seamlessly integrating Deep Learning capabilities into other software applications. Its main objective was to empower these software applications to effectively execute Deep Learning models. For that use case, JDLL was mainly oriented for application developers that wanted to incorporate DL methods in their softwares.

But after a couple of stable releases, the JDLL team noticed that because of its user-friendly interface, the library could be accessible not just to software developers but also to everyday users with basic scripting knowledge in languages like Python or Matlab. This makes it suitable for anyone interested in crafting processing routines that require Deep Learning models. Using a Java based or Java compatible scripting laguage such as Jython, one can easily use JDLL to run DL models.

Due to the dual possibilities of JDLL, there are 2 Quickstarts available depending on the interests of the user:

• Quickstart for developers if the user is interested in the integration of JDLL into their application.
• Quickstart for analysts/scripting if the user is interested in creating scripts in Jython that can run DL models easily to improve their processing routines. Not much experience with Python/Jython is required.

Examples

• ExampleLoadAndRunModel (PyTorch)
• ExampleLoadTensorflow1Tensorflow2
• ExampleDownloadEngine
• ExampleDownloadModel

Scripting examples

• example-run-model-in-fiji (Requires to be executed from Fiji, as it uses the application to display the result image, cannot be executed on Apple Silicon computers)
• example-run-pytorch-model-in-fiji.py (Requires to be executed from Fiji, as it uses the application to display the result image)
• example-download-bmz-model
• example-download-engine

Quickstart for developers

The next section (~10 min read) provides essential instructions for using JDLL divided in the following subsections:

• 0. Setting Up JDLL
• 1. Downloading a model (optional)
• 2. Installing DL engines
• 3. Creating the tensors
• 4. Loading the model
• 5. Running the model
• 6. Closing the model and the tensors

0. Setting Up JDLL

Download the dependency and include it in your project

In order to benefit from the library, include the dependency in your code. The dependency can be added manually or using a dependency manager such as Maven. If you are using Maven, add the following dependency to the project pom file:

<dependency>
  <groupId>io.bioimage</groupId>
  <artifactId>dl-modelrunner</artifactId>
  <version>0.4.0</version>
</dependency>

and add to <repositories> the following:

<repository>
  <id>scijava.public</id>
  <url>https://maven.scijava.org/content/groups/public</url>
</repository>

1. Downloading a model (optional)

If a model from the supported by JDLL is already available you can skip this step. Note that for Tensorflow the models need to be saved in the SavedModel format.

If no model is available, a good starting point is downloading any of the models of the Bioimage.io respository. The download can be done manually or using JDLL. Bioimag.io is seamlessly integrated into JDLL, offering multiple methods to effortlessly mange and use its models.

Here is an emaple of how JDLL can be used to download any Bioimage.io model, in this case the B. Sutilist bacteria segmentation - Widefield microscopy - 2D UNet model.

// Name of the model of interest, note that each model is unique. The names are case sensitive.