NavDP
Official implementation of the paper: "NavDP: Learning Sim-to-Real Navigation Diffusion Policy with Privileged Information Guidance"
Install / Use
/learn @InternRobotics/NavDPREADME
🔥 News
- We have open-sourced the entire deploy process based on LeKiwi, from hardware setup to algorithm deployment. 😺 Welcome to use it!
- We release the LoGoPlanner - a localization-grounded, end-to-end navigation framework.
- We release the InternVLA-N1 - the first end-to-end navigation dual-system.
- We release the InternNav - an all-in-one open-source toolbox for embodied naivgation.
🏡 Introduction
Navigation Diffusion Policy (NavDP) is an end-to-end mapless navigation model that can achieves cross-embodiment generalization without any real-world robot data. By building a highly efficient simulation data generation pipeline as well as the superior model design, NavDP achieves real-time path-planning and obstacle avoidance across various navigation tasks, including nogoal exploration, pointgoal navigation, imagegoal navigation.
<div style="text-align: center;"> <img src="./assets/images/teasor_method.png" alt="Dialogue_Teaser" width=100% > </div>💻 InternVLA-N1 System-1 Model
Please fill this form to access the link to download the latest model checkpoint.
🛠️ Installation
Please follow the instructions to config the environment for NavDP.
Step 0: Clone this repository
git clone https://github.com/InternRobotics/NavDP
cd NavDP/baselines/navdp/
Step 1: Create conda environment and install the dependency
conda create -n navdp python=3.10
conda activate navdp
pip install -r requirements.txt
🤖 Run NavDP Model
Run the following line to start navdp server:
python navdp_server.py --port ${YOUR_PORT} --checkpoint ${SAVE_PTH_PATH}
Then, follow the subsequent tutorial to build the environment for IsaacSim and start the evaluation in simulation. By running with our benchmark, you should be able to replicate the navigation examples below:
NoGoal Exploration
PointGoal Navigation
ImageGoal Navigation
🎢 InternVLA-N1 System-1 Benchmark
🏠 Overview
This repository is a high-fidelity platform for benchmarking the visual navigation methods based on IsaacSim and IsaacLab. With realistic physics simulation and realistic scene assets, this repository aims to build an benchmark that can minimizing the sim-to-real gap in navigation system-1 evaluation.
Highlights
- ⭐ Decoupled Framework between Navigation Approaches and Evaluation Process
The evaluation is accomplished by calling navigation method api with HTTP requests. By decoupling the implementation of navigation model with the evaluation process, it is much easier for users to evaluate the performance of novel navigation methods.
- ⭐ Fully Asynchronous Framework between Trajectory Planning and Following
We implement a MPC-based controller to constantly track the planned trajectory. With the asynchronous framework, the evaluation metrics become related to the navigation approaches' decision frequency which help align with the real-world navigation performance.
- ⭐ High-Quality Scene Asset for Evaluation
Our benchmark supports evaluation in diverse scene assets, including random cluttered environments, realistic home scenarios and commercial scenarios.
- ⭐ Support Image-Goal, Point-Goal and No-Goal Navigation Tasks
Our benchmark supports multiple navigation tasks, including no-goal exploration, point-goal navigation as well as image-goal navigation.
📋 Table of Contents
- 🏠 Overview
- 🌆 Prepare Scene Asset
- 🔧 Installation of Benchmark
- ⚙️ Installation of Baseline Library
- 💻 Running Basline as Server
- 🕹️ Running Teleoperation
- 📊 Running Evaluation
- 🔗 Citation
- 📄 License
- 👏 Acknowledgements
🌆 Prepare Scene Asset
Please download the scene asset from InternScene-N1 at HuggingFace. The episodes information can be directly accessed in this repo. After downloading, please organize the structure as follows:
assets/scenes/
├── SkyTexture/
│ ├── belfast_sunset_puresky_4k.hdr
│ ├── citrus_orchard_road_puresky_4k.hdr
│ ├── ...
├── Materials/
│ ├── Carpet
│ ├── textures/
│ ├── Carpet_Woven.mdl
│ └── ...
│ ├── ...
├── cluttered_easy/
│ └── easy_0/
│ ├── cluttered-0.usd/
│ ├── imagegoal_start_goal_pairs.npy
│ └── pointgoal_start_goal_pairs.npy
│ ├── ...
├── cluttered_hard/
│ └── hard_0/
│ ├── cluttered-0.usd/
│ ├── imagegoal_start_goal_pairs.npy
│ └── pointgoal_start_goal_pairs.npy
│ ├── ...
├── internscenes_commercial/
│ ├── models/
│ ├── Materials/
│ └── scenes_commercial/
│ ├── MV4AFHQKTKJZ2AABAAAAADQ8_usd/
│ ├── models/
│ ├── Materials/
│ ├── metadata.json
│ ├── start_result_navigation.usd
│ ├── imagegoal_start_goal_pairs.npy
│ └── pointgoal_start_goal_pairs.npy
│ ├── ...
├── internscene_home/
│ ├── models/
│ ├── Materials/
│ └── scenes_home/
│ ├── MV4AFHQKTKJZ2AABAAAAADQ8_usd/
│ ├── models/
│ ├── Materials/
│ ├── metadata.json
│ ├── start_result_navigation.usd
│ ├── imagegoal_start_goal_pairs.npy
│ └── pointgoal_start_goal_pairs.npy
│ ├── ...
| Category | Download Asset | Episodes | |------|------|-------| | SkyTexture | Link | - | | Materials | Link | - | | Cluttered-Easy | Link | Episodes | | Cluttered-Hard | Link | Episodes | | InternScenes-Home | Link | Episodes | | InternScenes-Commercial | Link | Episodes |
🔧 Installation of Benchmark
Our framework is based on IsaacSim 4.2.0 and IsaacLab 1.2.0, you can follow the instructions to configure the conda environment.
# create the environment
conda create -n isaaclab python=3.10
conda activate isaaclab
# install IsaacSim 4.2
pip install --upgrade pip
pip install isaacsim==4.2.0.2 isaacsim-extscache-physics==4.2.0.2 isaacsim-extscache-kit==4.2.0.2 isaacsim-extscache-kit-sdk==4.2.0.2 --extra-index-url https://pypi.nvidia.com
# check the isaacsim installation
isaacsim omni.isaac.sim.python.kit
# install IsaacLab 1.2.0
git clone https://github.com/isaac-sim/IsaacLab.git
cd IsaacLab/
git checkout tags/v1.2.0
# ignore the rsl-rl unavailable error
./isaaclab.sh -i
# check the isaaclab installation
./isaacla
