openvino2tensorflow

For those who lack skills in converting from ONNX to TensorFlow, I recommend using this tool. It is a tool in the making, so there are lots of bugs, but it is much easier than going through OpenVINO.

"Self-Created Tools to convert ONNX files (NCHW) to TensorFlow format (NHWC). The purpose of this tool is to solve the massive Transpose extrapolation problem in onnx-tensorflow (onnx-tf)."

https://github.com/PINTO0309/onnx2tf

<p align="center"> <img src="https://user-images.githubusercontent.com/33194443/194713389-62826ef1-0ea5-4b0e-acd4-540b9f5cf58f.png" /> </p>

---

---

---

<p align="center"> <img src="https://user-images.githubusercontent.com/33194443/104584047-4e688f80-56a5-11eb-8dc2-5816487239d0.png" /> </p>

This script converts the ONNX/OpenVINO IR model to Tensorflow's saved_model, tflite, h5, tfjs, tftrt(TensorRT), CoreML, EdgeTPU, ONNX and pb. PyTorch (NCHW) -> ONNX (NCHW) -> OpenVINO (NCHW) -> openvino2tensorflow -> Tensorflow/Keras (NHWC/NCHW) -> TFLite (NHWC/NCHW). And the conversion from .pb to saved_model and from saved_model to .pb and from .pb to .tflite and saved_model to .tflite and saved_model to onnx. Support for building environments with Docker. It is possible to directly access the host PC GUI and the camera to verify the operation. NVIDIA GPU (dGPU) support. Intel iHD GPU (iGPU) support.

Special custom TensorFlow binaries and special custom TensorFLow Lite binaries are used.

Work in progress now.

1. Environment

• Python 3.8+
• TensorFlow v2.10.0+
• PyTorch v1.12.1+
• TorchVision
• TorchAudio
• OpenVINO 2022.1.0
• TensorRT 8.4.0+
• trtexec
• pycuda 2022.1
• tensorflowjs
• coremltools
• paddle2onnx
• onnx
• onnxruntime-gpu (CUDA, TensorRT, OpenVINO)
• onnxruntime-extensions
• onnx_graphsurgeon
• onnx-simplifier
• onnxconverter-common
• onnxmltools
• onnx-tensorrt
• tf2onnx
• torch2trt
• onnx-tf
• tensorflow-datasets
• tf_slim
• edgetpu_compiler
• tflite2tensorflow
• openvino2tensorflow
• simple-onnx-processing-tools
• gdown
• pandas
• matplotlib
• paddlepaddle
• paddle2onnx
• pycocotools
• scipy
• blobconverter
• Intel-Media-SDK
• Intel iHD GPU (iGPU) support
• OpenCL
• gluoncv
• LLVM
• NNPACK
• WSL2 OpenCL

↥ Back to top

2. Use case

• PyTorch (NCHW) -> ONNX (NCHW) -> OpenVINO (NCHW) ->

  • -> openvino2tensorflow -> Tensorflow/Keras (NHWC/NCHW) -> TFLite (NHWC/NCHW)
  • -> openvino2tensorflow -> Tensorflow/Keras (NHWC/NCHW) -> TFJS (NHWC/NCHW)
  • -> openvino2tensorflow -> Tensorflow/Keras (NHWC/NCHW) -> TF-TRT (NHWC/NCHW)
  • -> openvino2tensorflow -> Tensorflow/Keras (NHWC) -> EdgeTPU (NHWC)
  • -> openvino2tensorflow -> Tensorflow/Keras (NHWC/NCHW) -> CoreML (NHWC/NCHW)
  • -> openvino2tensorflow -> Tensorflow/Keras (NHWC/NCHW) -> ONNX (NHWC/NCHW)
  • -> openvino2tensorflow -> Myriad Inference Engine Blob (NCHW)

• Caffe (NCHW) -> OpenVINO (NCHW) ->

  • -> openvino2tensorflow -> Tensorflow/Keras (NHWC/NCHW) -> TFLite (NHWC/NCHW)
  • -> openvino2tensorflow -> Tensorflow/Keras (NHWC/NCHW) -> TFJS (NHWC/NCHW)
  • -> openvino2tensorflow -> Tensorflow/Keras (NHWC/NCHW) -> TF-TRT (NHWC/NCHW)
  • -> openvino2tensorflow -> Tensorflow/Keras (NHWC) -> EdgeTPU (NHWC)
  • -> openvino2tensorflow -> Tensorflow/Keras (NHWC/NCHW) -> CoreML (NHWC/NCHW)
  • -> openvino2tensorflow -> Tensorflow/Keras (NHWC/NCHW) -> ONNX (NHWC/NCHW)
  • -> openvino2tensorflow -> Myriad Inference Engine Blob (NCHW)

• MXNet (NCHW) -> OpenVINO (NCHW) ->

  • -> openvino2tensorflow -> Tensorflow/Keras (NHWC/NCHW) -> TFLite (NHWC/NCHW)
  • -> openvino2tensorflow -> Tensorflow/Keras (NHWC/NCHW) -> TFJS (NHWC/NCHW)
  • -> openvino2tensorflow -> Tensorflow/Keras (NHWC/NCHW) -> TF-TRT (NHWC/NCHW)
  • -> openvino2tensorflow -> Tensorflow/Keras (NHWC) -> EdgeTPU (NHWC)
  • -> openvino2tensorflow -> Tensorflow/Keras (NHWC/NCHW) -> CoreML (NHWC/NCHW)
  • -> openvino2tensorflow -> Tensorflow/Keras (NHWC/NCHW) -> ONNX (NHWC/NCHW)
  • -> openvino2tensorflow -> Myriad Inference Engine Blob (NCHW)

• Keras (NHWC) -> OpenVINO (NCHW・Optimized) ->

  • -> openvino2tensorflow -> Tensorflow/Keras (NHWC/NCHW) -> TFLite (NHWC/NCHW)
  • -> openvino2tensorflow -> Tensorflow/Keras (NHWC/NCHW) -> TFJS (NHWC/NCHW)
  • -> openvino2tensorflow -> Tensorflow/Keras (NHWC/NCHW) -> TF-TRT (NHWC/NCHW)
  • -> openvino2tensorflow -> Tensorflow/Keras (NHWC) -> EdgeTPU (NHWC)
  • -> openvino2tensorflow -> Tensorflow/Keras (NHWC/NCHW) -> CoreML (NHWC/NCHW)
  • -> openvino2tensorflow -> Tensorflow/Keras (NHWC/NCHW) -> ONNX (NHWC/NCHW)
  • -> openvino2tensorflow -> Myriad Inference Engine Blob (NCHW)

• saved_model -> saved_model_to_pb -> pb

• saved_model ->

  • -> saved_model_to_tflite -> TFLite
  • -> saved_model_to_tflite -> TFJS
  • -> saved_model_to_tflite -> TF-TRT
  • -> saved_model_to_tflite -> EdgeTPU
  • -> saved_model_to_tflite -> CoreML
  • -> saved_model_to_tflite -> ONNX

• pb -> pb_to_tflite -> TFLite

• pb -> pb_to_saved_model -> saved_model

↥ Back to top

3. Supported Layers

• Currently, there are problems with the Reshape and Transpose operation of 2D,3D,5D Tensor. Since it is difficult to accurately predict the shape of a simple shape change, I have added support for forced replacement of transposition parameters using JSON files. #6-7-replace-weights-or-constant-values-in-const-op-and-add-transpose-or-reshape-or-cast-or-squeeze-or-unsqueeze-or-add-or-multiply-just-beforeafter-the-operation-specified-by-layer_id

  <details><summary>Supported Layers</summary><div>

  |No.|OpenVINO Layer|TF Layer|Remarks| |:--:|:--|:--|:--| |1|Parameter|Input|Convert to NHWC (Default) or NCHW| |2|Const|Constant, Bias|| |3|Convolution|Conv1D, Conv2D, Conv3D|Conv3D has limited support| |4|Add|Add|| |5|ReLU|ReLU|| |6|PReLU|PReLU|Maximum(0.0,x)+Minimum(0.0,alpha*x)| |7|MaxPool|MaxPool2D|| |8|AvgPool|AveragePooling1D, AveragePooling2D, AveragePooling3D|| |9|GroupConvolution|DepthwiseConv2D, Conv2D/Split/Concat|| |10|ConvolutionBackpropData|Conv2DTranspose, Conv3DTranspose|Conv3DTranspose has limited support| |11|Concat|Concat|| |12|Multiply|Multiply|| |13|Tan|Tan|| |14|Tanh|Tanh|| |15|Elu|Elu|| |16|Sigmoid|Sigmoid|| |17|HardSigmoid|hard_sigmoid|| |18|SoftPlus|SoftPlus|| |19|Swish|Swish|You can replace swish and hard-swish with each other by using the "--replace_swish_and_hardswish" option| |20|Interpolate|ResizeNearestNeighbor, ResizeBilinear|4D [N,H,W,C] or 5D [N,D,H,W,C]| |21|ShapeOf|Shape|| |22|Convert|Cast|| |23|StridedSlice|Strided_Slice|| |24|Pad|Pad, MirrorPad|| |25|Clamp|ReLU6, Clip|| |26|TopK|ArgMax, top_k|| |27|Transpose|Transpose|| |28|Squeeze|Squeeze|| |29|Unsqueeze|Identity, expand_dims|WIP| |30|ReduceMean|reduce_mean|| |31|ReduceMax|reduce_max|| |32|ReduceMin|reduce_min|| |33|ReduceSum|reduce_sum|| |34|ReduceProd|reduce_prod|| |35|Subtract|Subtract|| |36|MatMul|MatMul|| |37|Reshape|Reshape|| |38|Range|Range|WIP| |39|Exp|Exp|| |40|Abs|Abs|| |41|SoftMax|SoftMax|| |42|Negative|Negative|| |43|Maximum|Maximum|No broadcast| |44|Minimum|Minimum|No broadcast| |45|Acos|Acos|| |46|Acosh|Acosh|| |47|Asin|Asin|| |48|Asinh|Asinh|| |49|Atan|Atan|| |50|Atanh|Atanh|| |51|Ceiling|Ceil|| |52|Cos|Cos|| |53|Cosh|Cosh|| |54|Sin|Sin|| |55|Sinh|Sinh|| |56|Gather|Gather|| |57|Divide|Divide, FloorDiv|| |58|Erf|Erf|| |59|Floor|Floor|| |60|FloorMod|FloorMod|| |61|HSwish|HardSwish|x*ReLU6(x+3)*0.16666667, You can replace swish and hard-swish with each other by using the "--replace_swish_and_hardswish" option| |62|Log|Log|| |63|Power|Pow|No broadcast| |64|Mish|Mish|x*Tanh(softplus(x))| |65|Selu|Selu|| |66|Equal|equal|| |67|NotEqual|not_equal|| |68|Greater|greater|| |69|GreaterEqual|greater_equal|| |70|Less|less|| |71|LessEqual|less_equal|| |72|Select|Select|No broadcast| |73|LogicalAnd|logical_and|| |74|LogicalNot|logical_not|| |75|LogicalOr|logical_or|| |76|LogicalXor|logical_xor|| |77|Broadcast|broadcast_to, ones, Multiply|numpy / bidirectional mode, WIP| |78|Split|Split|| |79|VariadicSplit|Split, Slice, SplitV|| |80|MVN|reduce_mean, sqrt, reduce_variance|(x - reduce_mean(x)) / sqrt(reduce_variance(x) + eps)| |81|NonZero|not_equal, boolean_mask|| |82|ReduceL2|square, reduce_sum, sqrt|| |83|SpaceToDepth|SpaceToDepth|| |84|DepthToSpace|DepthToSpace|| |85|Sqrt|sqrt|| |86|SquaredDifference|squared_difference|| |87|FakeQuantize|subtract, multiply, round, greater, where, less_equal, add|| |88|Tile|tile|| |89|GatherND|gather_nd, reshape, cumprod, multiply, reduce_sum, gather, concat|| |90|NonMaxSuppression|non_max_suppression|WIP. Only available for batch size 1.| |91|Gelu|gelu||