Band

<div id="top"></div> <!-- PROJECT LOGO --> <br /> <div align="center"> <h2 align="center">Band: Multi-DNN Framework for Mobile-Cloud Platform </h2> </div>

Introduction

Band is an efficient deep learning platform for mobile-cloud collaborative support for multiple DNNs. Band supports backend-agnostic coordination of DNN requests on heterogeneous processors in a mobile device to <s>cloud server</s>. Band is currently backed by following backend machine learning frameworks.

| | Tensorflow v2.9.2 | Tensorflow v2.10.0 | ... | |---------|--------------------|--------------------|-----| | Android | ☑ | ☑ | | | iOS | ☐ | ☐ | | | gRPC | ☐ | ☐ | |

Band provides Java and C APIs, as well as an official plugin for Unreal Engine.

Useful Links

• Examples (requires update)
• Unreal Engine Plugin

Codebase

.
├── band
│   ├── backend  # backend-specific implementation of `interface`
│   ├── c  # C API
│   ├── docs
│   ├── interface  # Backend-agnostic interfaces. Each backend (e.g., Tensorflow Lite, MNN, ...) should implement them to communicate with Band core
│   ├── java  # Java API
│   ├── scheduler  # Schedulers
│   ├── test  # Test codes
│   ├── tool  # Benchmark tools
├── script  # Utilities
├── third_party
└── WORKSPACE

Getting Started

Prerequisites

• Install Android SDK 28, NDK v19.2.53456

  • or create Visual Studio Code Dev Container using [root]/.devcontainer
  • or utilize [root]/.devcontainer/Dockerfile

• Configure Android SDK, NDK for build system (Bazel)

  python configure.py
  

How to Build / Run

Refer to detailed instructions in [root]/script

• Run test for Android

  python script/run_test.py -android 
  

• Run test for Linux with GPU(OpenCL Support)

  python script/run_test.py -opencl 
  

• Build Android AAR

  sh script/build_aar_armv8.sh
  

• Build C API

  python script/build_c_api.py -android 
  

• Run benchmark -- check [root]/docs/benchmark.md

Citation

If you find our work useful, please cite our paper below! The original codebase for paper submission is archived here

@inproceedings{jeong2022band,
  title={Band: coordinated multi-DNN inference on heterogeneous mobile processors},
  author={Jeong, Joo Seong and Lee, Jingyu and Kim, Donghyun and Jeon, Changmin and Jeong, Changjin and Lee, Youngki and Chun, Byung-Gon},
  booktitle={Proceedings of the 20th Annual International Conference on Mobile Systems, Applications and Services},
  pages={235--247},
  year={2022}
}

<!-- ACKNOWLEDGMENTS -->

Acknowledgments

• Tensorflow
• NCNN - CPU affinity control

<p align="right">(<a href="#top">back to top</a>)</p>