OmniNxt

<div align="center"> <h2>OmniNxt: A Fully Open-source and Compact Aerial Robot with Omnidirectional Visual Perception</h2> <strong>IROS 2024 Oral</strong> <br> <a href="https://uav.hkust.edu.hk/current-members/" target="_blank">Peize Liu</a>, <a href="https://chen-albert-feng.github.io/AlbertFeng.github.io/" target="_blank">Chen Feng</a>^†, <a href="" target="_blank">Yang Xu</a>, <a href="" target="_blank">Yan Ning</a>, <a href="https://www.xuhao1.me/" target="_blank">Hao Xu</a>^†, and <a href="https://uav.hkust.edu.hk/group/" target="_blank">Shaojie Shen</a> <p> <h45> HKUST Aerial Robotics Group &nbsp;&nbsp; <br> </h5> ^†Corresponding Authors </p> <a href='https://arxiv.org/pdf/2403.20085.pdf'><img src='https://img.shields.io/badge/arXiv-OmniNxt-red' alt='arxiv'></a> <a href='https://hkust-aerial-robotics.github.io/OmniNxt/'><img src='https://img.shields.io/badge/Project_Page-OmniNxt-green' alt='Project Page'></a> <a href="https://www.bilibili.com/video/BV1AK421Y7Vz/?spm_id_from=333.999.0.0&vd_source=0af61c122e5e37c944053b57e313025a"><img alt="Bilibili" src="https://img.shields.io/badge/Video-Bilibili-blue"/></a> <!-- <a href="https://www.youtube.com/watch?v=jjPPIAAkPrk"><img alt="Youtube" src="https://img.shields.io/badge/Video-Youtube-red"/></a> --> </div> <div align="center"> <img src="misc/system_overview.png" alt="system overview" /> </div>

📢News

• [30/06/2024]: OmniNxt is accepted to IROS 2024.
• [15/08/2024] V0.1 Release

Outline

[TOC]

Please cite our paper if you use this project in your research:

• OmniNxt: A Fully Open-source and Compact Aerial Robot with Omnidirectional Visual Perception, Peize Liu, Chen Feng, Yang Xu, Yan Ning, Hao Xu, and Shaojie Shen, 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

@inproceedings{liu2024omninxt,
  title={Omninxt: A fully open-source and compact aerial robot with omnidirectional visual perception},
  author={Liu, Peize and Feng, Chen and Xu, Yang and Ning, Yan and Xu, Hao and Shen, Shaojie},
  booktitle={2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
  pages={10605--10612},
  year={2024},
  organization={IEEE}
}

Please kindly star ⭐️ this project if it helps you. We take great efforts to develop and maintain it 😁.

🤖Build your own OmniNxt

You should have basic knowledge of the standard quadrotor's electronic system.

BOM

| 🧰Component | 📏 Specification | 🔗 Purchase Link | | ------------------------------------- | ------------------------------------------------------------ | ------------------------------------------------------------ | | Quad-Fisheye Camera Set | | | | OAK-FFC-4P | Camera control board | Official / Taobao | | OAK-FFC-4P-new (Highly recommended! ) | Camera control board designed for OmniNxt | Official_Email / Taobao | | B0335 (OV9782) | Camera modules (Fisheye lens FOV larger than 210 degrees) | Official / Taobao | | Camera cables | Customized | Camera cable SCH | | Onboard computer | | | | Jetson Orin Nx | | | | Jetson Orin Nx carrier board | | DM | | Jetson Orin Nx radiator | | | | Jeston Orin Network card | | | | 2230 SSD | | | | Intel AX200 | | | | Flight platforms (6S) | | | | Nxt-FC | Project Page | MicoAir / Taobao | | Flight frame | Customized | Fusion360 | | Protector | Customized / Oddity RC XI35 protector | Fusion360 / Oddity RC | | Motors | 2204 1750KV/ 1804 2450KV | | | 6S Battery | | |

Our latest design can be accessed by Fusion360.🔗Link Access code: hkustUAV

Multi-fisheye Camera Module

ROS wrap driver 🔗oak_ffc_4p, which contains hardware information, ROS1 driver and test tools.

Calibration tool 🔗 quater-calib

Jetson Orin

Environment Info

We suggest you follow these settings to avoid conflict (especially CUDA and TensorRT).

Nxt-FC

We open-source hardware designs at 🔗NXT-FC. Nxt-FC supports PX4 and Ardupilot firmware.

Follow the instructions for setting up the Nxt-FC part to configure your flight controller.

The PX4 parameter of OmniNxt can be downloaded from Onedrive.

💾Runtime setup

Our Omni-VINS and Omni-Depth are developed from D2SLAM. Since building the Docker images on the local machine(Jetson Orin) usually takes a long time, we suggest you pull the images from the Docker Hub. If you want to build the Docker image on your local machine, follow the instructions [Build Docker Images On Local Machine](#Build Docker images in local machines).

the docker images structure is as follows:

Step 1 Clone the repository into your local machine

git clone --branch pr_fix_main https://github.com/HKUST-Aerial-Robotics/D2SLAM.git

Step 2 Configure your quad-cam parameters and VINS parameters following our template

cd ./D2SLAM/config/quadcam_drone_nxt_tmp

If you have already calibrated your quad fisheye camera set, you can simply replace the files with the same name, which includes:

1. stereo_calib_n_m_240_320.yaml (n and m is the camera number; used in Omni-Depth).
2. fisheye_cams.yaml. (used in Omni-VINS)

Step 3 Configure ./start_docker.sh script with your local environment.

./start_docker.sh is under the ./D2SLAM directory.

Modify the following parameters with the absolute path under your local host environment.

1. DATA_SET (where the data set is) (Optional)

Then run with the following command under the D2SLAM directory (very important, this will map your D2SLAM dir into the container)：

./start_docker.sh 1

Step 4 Launch algorithm modules

run all algorithm modules together (Omni-VINS & Omni-Depth)

## you should under /root/swarm_ws/
source ./devel/setup.bash
roslaunch d2vins quadcam.laucnh

Only launch Omni-VINS. Please remove the nodes in the red box

source ./devel/setup.bash
roslaunch d2vins quadcam.laucnh

Only launch Omni-Depth