TopDownPoseEstimation

What is this

TopDown Pose Estimation on iOS and PureSwift.

• BBox: Yolov7-tiny
• Pose Estimation: ViTPose

Install

$ git clone https://github.com/otmb/TopDownPoseEstimation.git
$ cd TopDownPoseEstimation/TopDownPoseEstimation
$ curl -OL https://github.com/mbotsu/KeypointDecoder/releases/download/0.0.1/vitpose-b256x192_fp16.mlmodel
$ curl -OL https://github.com/mbotsu/KeypointDecoder/releases/download/0.0.1/yolov7-tiny_fp16.mlmodel

Example

<img src="sample.png" width="500" />

COCO MS val2017

| Models | AP | |--|:--| | yolov7-tiny_fp16 + vitpose-b256x192_fp16.mlmodel | 0.589 | | yolov7-tiny_fp16 + vitpose_s256x192_wholebody_fp16.mlmodel | 0.579 | | yolov7-tiny_fp16 + vitpose_b256x192_wholebody_fp16.mlmodel | 0.600 |

Detail

Sample Models

| Models | Size | Keypoint | |--|--|:--:| | vitpose-b256x192_fp16.mlmodel | 172MB | 17 | | vitpose_s256x192_wholebody_fp16.mlmodel| 46.5MB | 133 | | vitpose_b256x192_wholebody_fp16.mlmodel | 172MB | 133 | | yolov7-tiny_fp16.mlmodel | 12.1MB | - |

COCO-Wholebody 133 When using Keypoint, change the following after introducing the model to the project.

Edit: PoseEstimation.swift

• keypointsNumber
• modelName

Create Model References

• ViTPose to CoreML
  • mbotsu/20221128_convert.ipynb
• Yolov7 to CoreML
  • john-rocky/CoreML-Models
    • Yolov7 Google Colab

References

• microsoft/human-pose-estimation.pytorch
• PaddlePaddle/PaddleDetection
• ViTAE-Transformer/ViTPose
• ViTPose to CoreML
  • mbotsu/20221128_convert.ipynb
• WongKinYiu/yolov7
• Yolov7 to CoreML
  • john-rocky/CoreML-Models
• AffineTransform
  • Perspective transform from quadrilateral to quadrilateral in Swift
• Drawing processing
  • Detecting human body poses in an image