<h1 align="center"> ASAP: Aligning Simulation and Real-World Physics for

Learning Agile Humanoid Whole-Body Skills </h1>

<div align="center">

Robotics: Science and Systems (RSS) 2025

[Website] [Arxiv] [Video]

<p align="center"> <img src="imgs/CMU-NV-logo-crop-png.png" height=50"> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </p>

<img src="https://agile.human2humanoid.com/static/images/asap-preview-gif-480P.gif" width="400px"/> </div> <!-- # Table of Contents -->

📚 Table of Contents

1. Overview

   • Links: Website • Arxiv • Video

2. Installation & Setup
   2.1 Base Frameworks
   2.2 IsaacGym Setup
   2.3 HumanoidVerse Setup
   2.4 IsaacSim + IsaacLab Setup
   2.5 Genesis Environment Setup

3. Training Pipelines
   3.1 Phase-Based Motion Tracking
   3.2 ASAP Delta Action Model
   - Train Delta Action Model
   - Finetune Policy with Delta Action Model

4. Motion Retargeting to Any Humanoid
   4.1 Step 1: SMPL Shape Preparation
   4.2 Step 2: SMPL Motion Preparation (AMASS)
   4.3 Step 3: Robot XML & Motion Config
   4.4 Step 4: Humanoid-SMPL Shape Fitting
   4.5 Step 5: Humanoid-SMPL Motion Retargeting

5. Deployment: Sim2Sim & Sim2Real
   5.1 Environment Setup
   5.2 Sim2Sim Deployment
   5.3 Sim2Real Deployment

6. Citation

7. License

TODO

• [x] Release code backbone
• [x] Release phase-based motion tracking training pipeline
• [x] Release ASAP motion datasets
• [x] Release motion retargeting pipeline
• [x] Release sim2sim in MuJoCo
• [x] Release sim2real with UnitreeSDK
• [x] Release ASAP delta action model training pipeline

Installation

ASAP codebase is built on top of HumanoidVerse (a multi-simulator framework for humanoid learning) and Human2Humanoid (our prior work on humanoid whole-body tracking).

HumanoidVerse allows you to train humanoid skills in multiple simulators, including IsaacGym, IsaacSim, and Genesis. Its key design logic is the separation and modularization of simulators, tasks, and algorithms, which enables smooth transfers between different simulators and the real world with minimum effort (just one line of code change). We leverage this framework to develop ASAP and study how to best transfer policies across simulators and the real world.

IsaacGym Conda Env

Create mamba/conda environment, in the following we use conda for example, but you can use mamba as well.

conda create -n hvgym python=3.8
conda activate hvgym

Install IsaacGym

Download IsaacGym and extract:

wget https://developer.nvidia.com/isaac-gym-preview-4
tar -xvzf isaac-gym-preview-4

Install IsaacGym Python API:

pip install -e isaacgym/python

Test installation:

python 1080_balls_of_solitude.py  # or
python joint_monkey.py

For libpython error:

• Check conda path:

  conda info -e
  

• Set LD_LIBRARY_PATH:

  export LD_LIBRARY_PATH=</path/to/conda/envs/your_env/lib>:$LD_LIBRARY_PATH
  

Install HumanoidVerse

Install dependencies:

pip install -e .
pip install -e isaac_utils
pip install -r requirements.txt

Test with:

HYDRA_FULL_ERROR=1 python humanoidverse/train_agent.py \
+simulator=isaacgym \
+exp=locomotion \
+domain_rand=NO_domain_rand \
+rewards=loco/reward_g1_locomotion \
+robot=g1/g1_29dof_anneal_23dof \
+terrain=terrain_locomotion_plane \
+obs=loco/leggedloco_obs_singlestep_withlinvel \
num_envs=1 \
project_name=TestIsaacGymInstallation \
experiment_name=G123dof_loco \
headless=False

<details> <summary>Note:</summary> This is ONLY for testing, NOT how we train the locomotion policy in the ASAP paper. But still, you can train a locomotion policy by:

HYDRA_FULL_ERROR=1 python humanoidverse/train_agent.py \
+simulator=isaacgym \
+exp=locomotion \
+domain_rand=NO_domain_rand \
+rewards=loco/reward_g1_locomotion \
+robot=g1/g1_29dof_anneal_23dof \
+terrain=terrain_locomotion_plane \
+obs=loco/leggedloco_obs_singlestep_withlinvel \
num_envs=4096 \
project_name=TestIsaacGymInstallation \
experiment_name=G123dof_loco \
headless=True \
rewards.reward_penalty_curriculum=True \
rewards.reward_initial_penalty_scale=0.1 \
rewards.reward_penalty_degree=0.00003 

</details>

IsaacLab Environment

Install IsaacSim

1. Download Omniverse Launcher
2. Install Isaac Sim through launcher
3. Set environment variables:

export ISAACSIM_PATH="${HOME}/.local/share/ov/pkg/isaac-sim-4.2.0"
export ISAACSIM_PYTHON_EXE="${ISAACSIM_PATH}/python.sh"

Install IsaacLab

git clone https://github.com/isaac-sim/IsaacLab.git
cd IsaacLab && ./isaaclab.sh --conda hvlab
mamba activate hvlab
sudo apt install cmake build-essential
./isaaclab.sh --install

Setup HumanoidVerse

pip install -e .
pip install -e isaac_utils

Genesis Environment

mamba create -n hvgen python=3.10
mamba activate hvgen
pip install genesis-world torch

Install dependencies:

pip install -e .
pip install -e isaac_utils

Motion Tracking Training

Train a phase-based motion tracking policy to imitate Cristiano Ronaldo's signature Siuuu move

python humanoidverse/train_agent.py \
+simulator=isaacgym \
+exp=motion_tracking \
+domain_rand=NO_domain_rand \
+rewards=motion_tracking/reward_motion_tracking_dm_2real \
+robot=g1/g1_29dof_anneal_23dof \
+terrain=terrain_locomotion_plane \
+obs=motion_tracking/deepmimic_a2c_nolinvel_LARGEnoise_history \
num_envs=4096 \
project_name=MotionTracking \
experiment_name=MotionTracking_CR7 \
robot.motion.motion_file="humanoidverse/data/motions/g1_29dof_anneal_23dof/TairanTestbed/singles/0-TairanTestbed_TairanTestbed_CR7_video_CR7_level1_filter_amass.pkl" \
rewards.reward_penalty_curriculum=True \
rewards.reward_penalty_degree=0.00001 \
env.config.resample_motion_when_training=False \
env.config.termination.terminate_when_motion_far=True \
env.config.termination_curriculum.terminate_when_motion_far_curriculum=True \
env.config.termination_curriculum.terminate_when_motion_far_threshold_min=0.3 \
env.config.termination_curriculum.terminate_when_motion_far_curriculum_degree=0.000025 \
robot.asset.self_collisions=0

After training, you can visualize the policy by:

python humanoidverse/eval_agent.py \
+checkpoint=logs/MotionTracking/xxxxxxxx_xxxxxxx-MotionTracking_CR7-motion_tracking-g1_29dof_anneal_23dof/model_5800.pt

This is the visualization of the policy after traning 5800 iters. The policy is able to imitate the motion of Cristiano Ronaldo's Siuuu move. With more training, the policy will be more accurate and smooth (see the video in the paper).

<img src="imgs/motion_tracking_5800.gif" width="400px"/>

ASAP delta action model training

Note that the only difference between the delta action model training and naive motion tracking training is that delta action model needs a motion file with extra keyname "action" in the motion file, so that the resulting RL policy we are training is able to use the delta action model to "control the robot" to match the real-world/sim2sim motions.

Train delta action model

python humanoidverse/train_agent.py \                                                                                   
  +simulator=isaacgym \
  +exp=train_delta_a_open_loop \
  +domain_rand=NO_domain_rand \
  +rewards=motion_tracking/delta_a/reward_delta_a_openloop \
  +robot=g1/g1_29dof_anneal_23dof \
  +terrain=terrain_locomotion_plane \
  +obs=delta_a/open_loop \
  num_envs=5000 \
  project_name=DeltaA_Training \
  experiment_name=openloopDeltaA_training \
  robot.motion.motion_file="<PATH_TO_YOUR_MOTION_FILE_WITH_ACTION_KEYNAME>" \
  env.config.max_episode_length_s=1.0 \
  rewards.reward_scales.penalty_minimal_action_norm=-0.1 \
  +device=cuda:0 \
  env.config.resample_motion_when_training=True \
  env.config.resample_time_interval_s=10000

Use delta action model for policy finetuning

HYDRA_FULL_ERROR=1 \
python humanoidverse/train_agent.py \
+simulator=isaacgym \
+exp=train_delta_a_closed_loop \
algo.config.policy_checkpoint='<PATH_TO_YOUR_DELTA_A_MODEL>' \
+domain_rand=NO_domain_rand_finetune_with_deltaA \
+rewards=motion_tracking/rewa