AirGym

<img src="https://img.shields.io/badge/PyTorch-2.0%2B-EE4C2C?logo=pytorch">

<div align="center"> <img src="doc/logo2.png" width="300" style="margin-bottom: 20px;"> <p style="font-size: 1.2em; margin-top: -10px;"> <i>A High-Performance Quadrotor Deep Reinforcement Learning Platform</i> <br> <b>Built on <span style="color: #76b900;">NVIDIA IsaacGym</span></b> </p> </div>

Changes Reminder

v0.5.1 released on 2025.09.27: Fix bugs: planning task resets at start when loading trained model planning_cnn_rate.pth.

v0.5.0 released on 2025.07.21: Update VAE model.

v0.4.0 released on 2025.04.21: Update reward functions for planning. Upload pretrained model for task planning.

v0.3.0 released on 2025.04.16: Bugs fixed on environments registry, and tests had done on every tasks.

~~v0.2.0 released on 2025.04.05~~: This version has bugs! Re-inplement the asset manager part to make it clear and efficient. Split the algorithm outside the /airgym.

v0.1.0dev released: first release.

<details> <summary style="display: list-item; cursor: pointer; text-decoration: none"> <h2 style="display: inline; border-bottom: none">Table of Contents</h2> </summary>

1. AirGym
2. Features
3. Environments

• Hovering
• Tracking
• Balloon (Target Reaching)
• Avoid (Dynamic Obstacle)
• Planning
• Customized Env

4. Installation

• Requirements
• One-shot Install
• Test the Installation

5. Usage

• Directory Structure Description
• Training from Scratch
• Playing and Testing
• Customize a New Task

6. TODO
7. FAQ
8. License
9. Acknowledgements
10. Citation

</details>

AirGym

AirGym is an open souce Python quadrotor simulator based on IsaacGym, a part of AirGym series Sim-to-Real workding flow. It provides a high-fidelity dynamics and Deep Reinforcement Learning (DRL) framework for quadrotor robot learning research. Furthermore, we also provide toolkits for transferring policy from AirGym simulator to the real quadrotor emNavi-X152b, making Sim-to-Real possible. The documentation website is at https://emnavi.tech/AirGym/.

<p align="center"> <img src="doc/pipeline-whitebg.png" alt="AirGym Sim-to-Real Pipeline"> </p>

The Sim-to-Real working flow of AirGym series is broken down into four parts:

• <span style="color: #76b900; ">AirGym</span>: the quadrotor simulation platform described in this repository, providing environments and basic training algorithm.
• <span style="color: #76b900; ">rlPx4Controller</span>: a flight geometric controller that maintains strict control logic alignment with the open-source PX4, for a better Sim-to-Real.
• <span style="color: #76b900; ">AirGym-Real</span>: an onboard Sim-to-Real module compatible with AirGym, enabling direct loading of pretrained models, supporting onboard visual-inertial pose estimation, ROS topic publishing, and one-shot scripted deployment.
• <span style="color: #76b900; ">control_for_gym</span>: a middleware layer based on MAVROS for forwarding control commands at various levels to PX4 autopilot. It includes a finite state machine to facilitate switching between DRL models and traditional control algorithms.

Features

AirGym is more lightweight and has a clearer file structure compared to other simulators, because it was designed from the beginning with the goal of achieving Sim-to-Real transfer.</p>

• Lightweight & Customizable: AirGym is extremely lightweight yet highly extensible, allowing you to quickly set up your own customized training task.
• Strict Alignment with PX4 Logic: Flight control in AirGym is supported by rlPx4Controller. It maintains strict control logic alignment with the open-source PX4, for a better Sim-to-Real.
• Multiple Control Modes: AirGym provides various control modes including PY (position & yaw), LV (linear velocity & yaw), CTA (collective thrust & attitude angle), CTBR (collective thrust & body rate), SRT (single-rotor thrust).
• Sim-to-Real Toolkits: AirGym series have complete flow of robot learning Sim-to-Real and provide a potential to transfer well-trained policies to a physical device.

Environments

AirGym provides four basic tasks <font face='courier new'>Hovering</font>, <font face='courier new'>Balloon</font>, <font face='courier new'>Tracking</font>, <font face='courier new'>Avoid</font>, and a higher level task <font face='courier new'>Planning</font>. All tasks are implemented on X152b quadrotor frame since this is our Sim-to-Real device.

<font face='courier new'>Hovering</font>

Task <font face='courier new'>Hovering</font>: the quadrotor is expected to be initialized randomly inside a cube with a side length of 2 meters, then converge into the center and hover until the end of the episode. Also, this task can be used as "go to waypoint" task if specify a target waypoint.

<p align="center"> <img src="doc/hovering1.gif" alt="Demo Video" /> </p>

<font face='courier new'>Balloon</font>

Task <font face='courier new'>Balloon</font>: also called target reaching. It is essentially a variation of the hovering task, but with a key difference: the quadrotor moves at a higher speed, rapidly dashing toward the balloon (the target).

<p align="center"> <img src="doc/balloon-hitting1.gif" alt="Demo Video" /> </p>

<font face='courier new'>Tracking</font>

Task <font face='courier new'>Tracking</font>: tracking a sequance of waypoints which is played as a trajectory. The tracking speed is effected by the trajectory playing speed.

<p align="center"> <img src="doc/tracking1.gif" alt="Demo Video" /> </p>

<font face='courier new'>Avoid</font>

Task <font face='courier new'>Avoid</font>: hover and try to avoid a cube or a ball with random throwing velocity and angle. This task provides depth image as input.

<p align="center"> <img src="doc/avoid1.gif" alt="Demo Video" /> </p>

<font face='courier new'>Planning</font>

Task <font face='courier new'>Planning</font>: a drone navigates and flies through a random generated woods like area, using only depth information as input. No global information is utilized in this task which means a better adaptation in a GNSS-deny environment and without a VIO.

<p align="center"> <img src="doc/planning-sim.gif" alt="Demo Video" /> </p>

Customized Environments

Furthermore, you can build a customized environment and even task upon AirGym. Here is a simple demo of random assets generation.

<p align="center"> <img src="doc/airgym_demo1.gif" alt="Demo Video" /> </p>

Installation

Requirements

1. Ubuntu 20.04
2. Conda or Miniconda

Note: this repository has been tested on Ubuntu 20.04 PyTorch 2.0.0 + CUDA11.8 with RTX4090.

One-shot Install

Run configuration.sh to download and install NVIDIA Isaac Gym Preview 4 and rlPx4Controller at ~, and create a new conda environment named airgym. Note that sudo is required to install apt package:

chmod +x configuration.sh
./configuration.sh

Test the Installation

You can run the example script which is a quadrotor position control illustration of 4096 parallel environments:

conda activate airgym
python airgym/scripts/example.py --task hovering --ctl_mode pos --num_envs 4

The default ctl_mode is position control.

Usage

Directory Structure Parsing

AirGym/
│
├── airgym/                                      # source code
│   ├── envs/                                    # envs definition
│   │    ├── base/                               # base envs
│   │    └── task/                               # tasks
│   │          ├── planning.py                   # task env file
│   │          ├── planning_config.py            # task env config file
│   │          ├── ...
│   │           ...
│   │
│   ├── assets/                                  # assets
│   │     ├── env_assets/                        # env assets
│   │     └── robots/                            # robot assets
│   ├── scripts/                                 # debug scripts
│   └── utils/                                   # others
│
├── doc/                                         # doc assets
│
├── lib/                                         # RL algorithm
│
├── scripts/                                     # program scipts
│   ├── config/                                  # config file for training
│   ├── runner.py                                # program entry
│   └── ...                                      
│
└── README.md                                    # introduction

Training from Scratch

We train the model by a rl_games liked customed PPO (rl_games was discarded after the version 0.0.1beta because of its complexity for use). The training algorithm is at airgym/lib modified from rl_games-1.6.1. Of course you can use any other RL libs for training.

Training from scratch can be down within minites. The