TransferCL

Table of Contents

• 1. Why TransferCL ?
• 2. How does it work?
  • 2.1 Transfer Learning
• 3. Installation
  • 3.1 Installation from prebuild packages
  • 3.2 Building from source: Native Library installation
    • 3.1.1 Pre-requisites
      • 3.1.1.1 Where to find the appropriated OpenCL shared-library
    • 3.1.2 Procedure
    • 3.1.3 Android application installation
• 4. How to use it
• 5. Case study
• 6. Important remarks
• 7. How to see the output
• 8. To get in contact
• 9. Contribute

TransferCL is an open source deep learning framework which has been designed to run on mobile devices. The goal is to enable mobile devices to tackle complex deep learning oriented-problem reserved to desktop computers. This project has been initiated by the parallel and distributed processing laboratory at National Taiwan University. Olivier Valery develloped this tool during his PhD at National Taiwan University. TransferCL is released under Mozilla Public Licence 2.0.

1. Why TransferCL ?

Recent mobile devices are equipped with multiple sensors, which can give insight into the mobile users' profile. We believe that such information can be used to customize the mobile experience for a specific user.

The primary goal of TransferCL is to leverage Transfer Learning on mobile devices. Our work is based on the DeepCL Library. Despite their similarities, TransferCL has been designed to run efficiently on a broad range of mobile devices. As a result, TransferCL implements its own memory management system and own OpenCL kernels in order take into account the specificity of mobile devices' System-on-Chip.

2. How does it work?

TransferCL is implemented in C++ and is able to run on any Android device with an OpenCL compliant GPU (the vast majority of modern Android devices). TransferCL provides several APIs which allow programmers to transparently leverage deep learning on mobile devices.

2.1 Transfer Learning

Modern mobile devices suffer from two major issues that prevent them from training a deep neural network on mobile devices via a classic supervised learning approach:

• First, the computing capabilities are relatively limited in comparison to servers.
• Then, a single mobile device may not have a sufficient label data in its training data set to train a deep neural network accurately.

Transfer learning is a technique that shortcuts a lot of this work by taking a fully-trained model for a set of categories like ImageNet and retraining from the existing weights for new classes. The use of pre-trained features is currently the most straightforward and most commonly way to perform transfer learning, but it is not the only one.

For more information, please check these websites:

• Transfer Learning with TensorFlow
• Transfer Learning with CNTK
• A survey of existing Transfer Learning techniques
• The class Convolutional Neural Networks for Visual Recognition at Stanford
• A paper study of transfer learning performance

3. Installation

There are two ways to install TranferCL:

1. From the source
   • This method enables the developer to build TranferCL for any particular mobile device architecture. We recommend this approach.
2. Importing TranferCL from our prebuilt directory
   • TransferCL has been pre-build for several commonly used hardware configurations. For these configurations, the shared-library can be imported directly in the Android application. However, we emphasize that once built, a shared-library is specific to a CPU ABI (armeabi-v7a, arm64-v8a ...) a GPU architecture (Adreno, Mali ...) and won't work for any other configurations than the one targeted initially.

3.1 Installation from prebuild packages

• In the folder prebuild library, you can find the binary files (to include in your Android aplication) and the JavaWrapprer.
• In this folder, this file includes more details about their utilization.

3.2 Building from source: Native Library installation

3.1.1 Pre-requisites

• OpenCL compliant GPU, along with appropriate OpenCL driver:

  • The libOpenCL.so, corresponding to the mobile device's GPU which is being targeting, need to to be placed in the folder extra_libs.
  • the headers files (*.h) need to be placed in the folder include

• CrystaX NDK:

  • Google NDK provides a set of tools to build native applications on Android. Our work is based on CrystaX NDK, which has been developed as a drop-in replacement for Google NDK. For more information, please check their website.
  • It is still possible to use Google NDK, however, the user will need the import Boost C++ by himself.

3.1.1.1 Where to find the appropriated OpenCL shared-library

As mentioned previously, the installation of TransferCL requires the compatible libOpenCL.so library and the corresponding OpenCL headers:

• The headers: the simplest way is extracting them from Adreno/Mali SDK. For Adreno SDK, they can be found at <Adreno_SDK>/Development/Inc/CL. For Mali SDK, they can be found at <MALI_SDK>/include/CL.
• The libOpenCL.so: the library is generally already present on the mobile device and can be pulled via adb pull /system/vendor/lib/libOpenCL.so .. (the path may change from one brand to another)

3.1.2 Procedure

• git clone https://github.com/OValery16/TransferCL.git
• add your libOpenCL.so in the folder extra_libs.
• add the OpenCL header in the folder include.

Your repository should look like that:

• In the folder 'jni', create a \*.cpp file and a &ast.h file, whose role is to interface with TranferCL. The Android application will call this file's method to interact with the deep learning network.
  • An example can be found in transferCLinterface.cpp
  • The name of the functions need to be modified in order to respect the naming convention for native function in NDK/JNI application: Java_{package_and_classname}_{function_name}(JNI arguments)
    • For example the Java_com_sony_openclexample1_OpenCLActivity_training means that this method is mapped to the training method from the OpenCLActivity activity in the com.sony.openclexample1 package.
    • For more information about this naming convention, please check this website
• In the 'Android.mk', change the line LOCAL_SRC_FILES :=transferCLinterface.cpp to LOCAL_SRC_FILES :={your_file_name}.cpp (replace 'your_file_name' by the name of the file you just created)
• In the 'Application.mk' change the line APP_ABI := armeabi-v7a to APP_ABI := {the_ABI_you_want_to_target} (replace 'the_ABI_you_want_to_target' by the ABI you want to target)
  • A list of all supported ABIs is given on the NDK website.
  • Make sure that your device supports the chosen ABI (otherwise it won't be able to find TransferCL 's methods). If you are not certain, you can check which ABIs are supported by your device, via some android applications like OpenCL-Z.
• Run CrystaX NDK to build your shared library with the command ndk-build (crystax-ndk-X\ndk-build where X is CrystaX NDK version)

>ndk-build

• CrystaX NDK will output several shared library files (they are specific to your mobile device ABIs)

3.1.3 Android application installation

• Create your Android project.
• Don't forget to respect the name conversion that you chose earlier (otherwise your application won't find your native methods)
• In your activity, you have to load your native library as following

    static {
      try {
          System.loadLibrary("openclexample1");  //Just put your libaries name
      }
      catch (UnsatisfiedLinkError e) {
          sfoundLibrary = false;
      }
    }

• Define the methods that have been implemented natively (in the shared library) as in the example

    //the name need to be the same as the one defined in the shared library
    public static native int training(String path, String cmdTrain);
    public static native int prediction(String path, String cmdPrediction);
    public static native int prepareFiles(String path, String fileNameStoreData,String fileNameStoreLabel, String fileNameStoreNormalization, String manifestPath, int nbImage, int imagesChannelNb);

• Build your applications

4. How to use it

• In the folder study case, you can find a template application using TranferCL. This application defines 2 Java source package:
  • com.transferCL, which is a java wrapper for the native methods defined in TranferCL (TransferCLlib.java).
  • com.example.myapplication, which is an android activity (```MainActivit