SysMocap

<h1 align="center"> <img src="https://github.com/xianfei/SysMocap/assets/8101613/adca7a3c-bdb2-4bda-af26-7ef9ba218c4c" align="center" height="128px" width="128px">

SysMocap

</h1> <p align="center"> <a href="https://github.com/xianfei/SysMocap/actions" target="_blank"> <img src="https://github.com/xianfei/SysMocap/actions/workflows/main.yml/badge.svg" alt="GitHub Actions" /> </a> <a href="https://github.com/xianfei/SysMocap/releases" target="_blank"> <img src="https://badgen.net/github/release/xianfei/SysMocap?color=cyan" alt="release" /> </a> <a href="#" target="_blank"> <img src="https://badgen.net/github/forks/xianfei/SysMocap" alt="forks" /> </a> <a href="#" target="_blank"> <img src="https://badgen.net/github/stars/xianfei/SysMocap?color=yellow" alt="stars" /> </a> </p> <p align="center"> English Version | <a href="./README.zh-cn.md">中文版本</a> </p>

A cross-platform real-time video-driven motion capture and 3D virtual character rendering system for VTuber/Live/AR/VR.

Available for Windows, macOS (including Apple Silicon & Intel) & Linux (source code only)

Download Now

(This is a multi-language software, including English and Chinese. )

Highlights

🌟 Beautiful GUI with Material Design 3 Color System, with Dark Mode supported.

🌟 Easy to used. You can import 3D models just with drags.

https://user-images.githubusercontent.com/8101613/167257555-8b8d4b99-f99f-4b79-8891-967b8723e3f8.mp4

🌟 Support WebXR API on Mocap Forwarding (HTTPS only)

https://user-images.githubusercontent.com/8101613/167257906-596919a5-4c0e-4795-865f-384a15c0d39f.mp4

🌟 Model viewer with bones & dressing controller

🌟 Support OBS live-streaming

🌟 Support full-body motion capture

🌟 Support Auto Skeleton Detection for All VRM files (including VRM 0.x and VRM 1.0) and Mixamo Format FBX files

🌟 Support Any Skeleton Structure with Manual Mapping

🌟 You can use VRoid Studio to create your avatar very easily. Supporting VRM 0.x & 1.0!

<img width="1492" alt="vroid" src="https://github.com/xianfei/SysMocap/assets/8101613/8847e960-3145-4ee5-b512-53ae8ec74901">

More Effect Demonstration

🌟 Facial

🌟 Half-body

🌟 Half-body with Hands

🌟 Full-body

System architecture

How to Download prebuilt package (Windows & macOS):

Download Now

1. Windows Portable Version: Simply extract the archive and run SysMocap.exe.

• SysMocap-Windows-x64-<version>.7z: Windows 64-bit version, suitable for x86_64 processors (AMD, Intel, etc.) running 64-bit Windows 10 & 11.

• SysMocap-Windows-arm64-<version>.7z: Windows ARM 64-bit version, suitable for ARM64 processors (such as Qualcomm Snapdragon) running 64-bit Windows 10 & 11.

2. Windows Installer: Double-click to install.

<img width="1317" alt="Snipaste_2024-07-07_20-56-11" src="https://github.com/xianfei/SysMocap/assets/8101613/8aa6fda0-5963-41dc-bad9-e28b6a2d7619">

• SysMocap-Windows-x64-installer-<version>.msi: Windows 64-bit version, suitable for x86_64 processors (AMD, Intel, etc.) running 64-bit Windows 10 & 11.

• SysMocap-Windows-arm64-installer-<version>.msi: Windows ARM 64-bit version, suitable for ARM64 processors (such as Qualcomm Snapdragon) running 64-bit Windows 10 & 11.

3. macOS DMG Image: Drag SysMocap.app to the Applications folder.

<img width="600" alt="Snipaste_2024-07-07_20-56-11" src="https://github.com/xianfei/SysMocap/assets/8101613/7a47820d-5d7c-421f-822e-d02bad2d6f29">

• SysMocap-macOS-x64-<version>.dmg: Suitable for Apple computers with Intel chips and Hackintosh devices, running macOS 10.15 or later.

• SysMocap-macOS-arm64-<version>.dmg: Suitable for Apple computers with M series chips (Apple Silicon).

Note for user on macOS:

• You need set Gatekeeper to Anywhere in System Settings (using sudo spctl --master-disable) <img width="478" alt="image" src="https://github.com/xianfei/SysMocap/assets/8101613/7b747e44-789c-4a61-83d7-c8e784a14856">

• If you got “SysMocap” is damaged and can’t be opened. You should move it to the Trash. Please run sudo xattr -r -d com.apple.quarantine /Applications/SysMocap.app in your terminal

How to run from source code (need lastest Node.js):

git clone https://github.com/xianfei/SysMocap.git
cd SysMocap
npm i
npm start

Notice

1. HTTP & HTTPS will use same port in Mocap Data Forward.

Required Skeleton Node in glTF/glb/FBX Model File for Macap:

(If not same as above, you need rebind them manually.)

• Hips (Main Node, both Position and Rotation. Ratation only for other nodes)

• Neck

• Chest

• Spine

• RightUpperArm

• RightLowerArm

• LeftUpperArm

• LeftLowerArm

• LeftUpperLeg

• LeftLowerLeg

• RightUpperLeg

• RightLowerLeg

Star History

Improve Translate

If you want to contribute language translation, you can add translation in utils/language.js

Then modify mainview/framework.html

 <div class="settings-item">
  <i class="mdui-icon material-icons" style="margin-right: 10px; margin-top: -3px">language</i>
  <span>{{language.tabSettings.ui.language}}</span>
  <select v-model="settings.ui.language" class="mdui-select" style="float: right; margin-right: 10px" id="demo-js-2">
    <option value="zh">简体中文</option>
    <option value="en">English</option>
  </select>
</div>

Fonts

Nowar Rounded for East Asian languages (汉语、漢語、日本語、한국어)

Quicksand for English and etc.

Thanks

You can see all dependencies in package.json

• google/mediapipe/Holistic for Mocap

• kalidokit for Calulate Mocap Data

• electron and Vue.js for GUI Framework

• Material color utilities for Color Picking

Cite

@INPROCEEDINGS{9974484,
  author={Song, Wenfeng and Wang, Xianfei and Gao, Yang and Hao, Aimin and Hou, Xia},
  booktitle={2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)}, 
  title={Real-time Expressive Avatar Animation Generation based on Monocular Videos}, 
  year={2022},
  volume={},
  number={},
  pages={429-434},
  doi={10.1109/ISMAR-Adjunct57072.2022.00092}}