Learning Analytical Posterior Probability for Human Mesh Recovery

Project Page | Paper | 推文

Learning Analytical Posterior Probability for Human Mesh Recovery
Qi Fang, Kang Chen, Yinghui Fan, Qing Shuai, Jiefeng Li, Weidong Zhang
CVPR 2023

<div align="left"> <img src="assets/soccer.gif" height="200"> <img src="assets/parkour.gif" height="200"> <img src="assets/samples.png" height="200"> <br> <sup> Single-frame results. (@<a href="https://www.youtube.com/watch?v=hS3UGaZSAuk&ab_channel=Joris">soccer</a>, @<a href="https://www.youtube.com/watch?v=uqOWoeJ7bpo&ab_channel=urbanamadei">parkoour</a>) </sup> </div>

Installation

conda create -n propose python=3.8
conda activate propose
pip install -r requirements.txt
# visualization
pip install opendr

See this issue if errors occur when installing opendr.

Data

SMPL Files and Pretrained Model

Create a folder called model_files.
Download smpl files from here or the official website. Unzip it and place it at model_files/.
Download the pretrained model and place it at model_files/.

Training Dataset

Download relevant datasets, including but not limited to Human3.6M, MPI-INF-3DHP, MSCOCO, 3DPW, EFT, AGORA, COCO-WholeBody.

We prepare the data following HybrIK. Furthermore, the details of annotation structure can be found in dataset.

Quick Demo

python scripts/demo.py --img-dir ./examples --out-dir dump_demo --ckpt './model_files/propose_hr48_xyz.pth'

Train

# The distributed training in scripts/train.sh can be opened if multiple nodes are available.
sh ./scripts/train.sh exp_name ./configs/smpl_hm_xyz.yaml

Discussion

See discussion for clarifications and limitations. Please open an issue (preferred) or send an email to Qi Fang (fangqi19@zju.edu.cn) if you have any questions.

TODO

[ ] Evaluation part (original).
[ ] Saved training annotations.
[ ] Distribution visualization.

Citation

@inproceedings{fang2023propose,
  title     = {Learning Analytical Posterior Probability for Human Mesh Recovery},
  author    = {Fang, Qi and Chen, Kang and Fan, Yinghui and Shuai, Qing and Li, Jiefeng and Zhang, Weidong},
  booktitle = {CVPR},
  year      = {2023},
}

Acknowledgement

We would like to thank Dingwei Wu for the advice about rendering in Blender, Chi Zhou and Yanjun Li for the discussion. Some of the code is adapted from HybrIK, HierarchicalProbabilistic3DHuman, SMPLX, SPIN, MeTRAbs, HRNet.

ProPose

Install / Use

README