SLAMesh
[ICRA 2023] SLAMesh: Real-time LiDAR Simultaneous Localization and Meshing
Install / Use
/learn @lab-sun/SLAMeshREADME
SLAMesh
SLAMesh: Real-time LiDAR Simultaneous Localization and Meshing
This is the code of SLAMesh.
You are welcome to visit my personal repository and raise issues there so that I can receive immediate notifications. I may also update it more frequently but sometimes the result may slightly differ from the paper.
Update
17/Aug/2023, Code released. Feel free to contact me with any questions. We are confident that this work introduced a novel approach to LiDAR SLAM, and we welcome everyone to explore opportunities in this approach. :two_men_holding_hands:
10/Mar/2023, Preprint of our paper can be found on: paper, slides.
16/Jan/2023, The paper has been accepted for presentation on ICRA 2023.
15/Sep/2022, The paper has been submitted to ICRA 2023.
1. Introduction
This work designs a Simultaneously Localization And Meshing system (SLAMesh). Mesh is a lightweight 3-D dense model. It can model complex structures and has the feasibility for rendering. We bridge localization with meshing at the same time to benefit each other.
1.1 Main features
- Build, register, and update the mesh maps directly in real-time with CPU resources, around 40 Hz.
- Provide accurate odometry. Kitti odometry benchmark: 0.6763%.
- Established a new approach to LiDAR SLAM different from point-cloud, NDT, and surfel map SLAM.
- Continuous mapping with uncertainty via a reconstruction. Fast meshing, matching without kd-tree.
<img src="https://github.com/RuanJY/SLAMesh/blob/master/fig/slamesh_real_world_test_2.gif" width="48%" alt="slamesh_gif_real_world2"> <img src="https://github.com/RuanJY/SLAMesh/blob/master/fig/slamesh_kitti07.gif" width="48%" alt="slamesh_gif_kitti07">
</div>1.2 Demo video
On public dataset:
<div align=center> <a href="https://www.youtube.com/watch?v=bm9u0-d4giw" target="_blank"><img src="https://github.com/RuanJY/SLAMesh/blob/master/fig/cover.png" alt="video1" width="85%" /></a> </div>(or watch it on bilibili)
On self-collected dataset:
<div align=center> <a href="https://www.youtube.com/watch?v=-zMNndGmUho" target="_blank"><img src="https://github.com/RuanJY/SLAMesh/blob/master/fig/real_word_cover2.png" alt="video2" width="85%" /></a> </div>(or watch it on bilibili)
1.3 Find more detail
If you find our research helpful to your work, please cite our paper:
[1] Jianyuan Ruan, Bo Li, Yibo Wang, and Yuxiang Sun, "SLAMesh: Real-time LiDAR Simultaneous Localization and Meshing" ICRA 2023 (pdf, IEEE, slides.).
@INPROCEEDINGS{10161425,
author={Ruan, Jianyuan and Li, Bo and Wang, Yibo and Sun, Yuxiang},
booktitle={2023 IEEE International Conference on Robotics and Automation (ICRA)},
title={SLAMesh: Real-time LiDAR Simultaneous Localization and Meshing},
year={2023},
volume={},
number={},
pages={3546-3552},
doi={10.1109/ICRA48891.2023.10161425}}
Other related papers:
[2] Jianyuan Ruan, Bo Li, Yinqiang Wang and Zhou Fang, "GP-SLAM+: real-time 3D lidar SLAM based on improved regionalized Gaussian process map reconstruction," IROS 2020. link.
[3] Bo Li, Yinqiang Wang, Yu Zhang. Wenjie Zhao, Jianyuan Ruan, and Pin Li, "GP-SLAM: laser-based SLAM approach based on regionalized Gaussian process map reconstruction". Autonomous Robot 2020.link
If you understand Chinese, you can also refer to my Master's thesis, an article on the WeChat platform: SLAMesh: 实时LiDAR定位与网格化模型构建 , and a talk in 自动驾驶之心and计算机视觉life.
2. Build
2.1 Prerequisite
We tested our code in Ubuntu18.04 with ROS melodic and Ubuntu20.04 with ROS neotic.
ROS
Install ros following ROS Installation. We use the PCL and Eigen library in ROS.
Ceres
Install Ceres Solver, version 2.0 or 2.1. follow Ceres Installation.
Notice that version > 2.1 may have some compatibility issues with our code. So you can use following command to install ceres:
apt-get install cmake libgoogle-glog-dev libgflags-dev libatlas-base-dev libsuitesparse-dev -y
git clone https://github.com/ceres-solver/ceres-solver.git -b 2.1.0
mkdir ceres-bin && cd ceres-bin && cmake .. && make -j$(($(nproc)-2))
make install
mesh_tools
We use mesh_tools to visualize the mesh map with the mesh_msgs::MeshGeometryStamped ROS message. mesh_tools also incorporates navigation functions upon mesh map. Mesh tool introduction
Install mesh_tools by:
- Install lvr2:
sudo apt-get install build-essential \
cmake cmake-curses-gui libflann-dev \
libgsl-dev libeigen3-dev libopenmpi-dev \
openmpi-bin opencl-c-headers ocl-icd-opencl-dev \
libboost-all-dev \
freeglut3-dev libhdf5-dev qtbase5-dev \
qt5-default libqt5opengl5-dev liblz4-dev \
libopencv-dev libyaml-cpp-dev
If in Ubuntu18.04, use libvtk6 because libvtk7 will conflict with pcl-ros in melodic.
sudo apt-get install libvtk6-dev libvtk6-qt-dev
Else if, in Ubuntu 20.04,
sudo apt-get install libvtk7-dev libvtk7-qt-dev
End of IF
then:
cd a_non_ros_dir
build:
git clone https://github.com/uos/lvr2.git
cd lvr2
mkdir build && cd build
cmake .. && make
sudo make install
It may take you some time.
- Install mesh_tools, (I can not install it from official ROS repos now, so I build it from source)
cd slamesh_ws/src
git clone https://github.com/naturerobots/mesh_tools.git
cd ..
rosdep update
rosdep install --from-paths src --ignore-src -r -y
catkin_make
source devel/setup.bash
2.2 SLAMesh
Clone this repository and build:
cd slamesh_ws/src
git clone https://github.com/RuanJY/SLAMesh.git
cd .. && catkin_make
mkdir slamesh_result
source ~/slamesh_ws/src/devel/setup.bash
2.3 Docker support
If you encounter some troubles when building slamesh, the problem may lay down on the prerequisite; we advise you to use the docker image:
Option 1, Build the docker image use Dockerfile:
cd slamesh_ws/src/SLAMesh/docker/
chmod +x run_docker.sh
./run_docker.sh -b
Run the docker image:
docker run -it --rm --gpus=all --runtime=nvidia -e NVIDIA_DRIVER_CAPABILITIES=all --env="DISPLAY=$DISPLAY" \
-e "QT_X11_NO_MITSHM=1" --volume="/tmp/.X11-unix:/tmp/.X11-unix:rw" --env="XAUTHORITY=$XAUTH" --volume="$XAUTH:$XAUTH" \
--net=host --privileged \
--volume=path_of_dataset_in_your_PC:/home/ruanjy/Dataset/ \
--volume=/home/$USER/slamesh_result:/home/slamesh/slamesh_ws/slamesh_result \
--name=test_slamesh slamesh bash
change the path_of_dataset_in_your_PC to the folder path of the dataset in your PC, like --volume=/home/ruanjy/kitti_odometry/data_odometry_velodyne/dataset/:/home/ruanjy/Dataset/ \
In this way, SLAMesh are built already, so source the workspace and run the launch file. You can also play rosbag in your host PC.
Option 2, otherwise, you can also pull the pre-built docker image with VNC directly:
docker pull pleaserun/rjy_slam_work:slamesh_18.04
Run the docker image:
docker run -it -p 5900:5900 -p 2222:22 -e RESOLUTION=1920x1080 \
--volume=path_of_dataset_in_your_PC:/home/ruanjy/Dataset/ \
--volume=/home/$USER/slamesh_result:/home/slamesh/slamesh_ws/slamesh_result \
--name test_slamesh pleaserun/rjy_slam_work:slamesh_18.04
change the path_of_dataset_in_your_PC to the folder path of the dataset in your PC, like --volume=/home/ruanjy/kitti_odometry/data_odometry_velodyne/dataset/:/home/ruanjy/Dataset/ \
In this way you can use VNC to enter a graphical interface via port 5900, or use ssh to connect container via port 2222.
Move to the dictionary slamesh_ws/src, and complete step from 2.2.
3. Usage
3.1 Run kitti dataset:
The dataset is available at KITTI dataset.
Set the parameter data_path in slamesh_kitti.launch to your folder of kitti dataset path
The file tree should be like this:
file_loc_dataset
├── 00
| └──velodyne
| ├── 000000.bin
| └── ...
└──01
For example, if you want to run the 07 sequence:
roslaunch slamesh slamesh_kitti_meshing.launch seq:=/07
You should get:
<div align=center> <img src="https://github.com/RuanJY/SLAMesh/blob/master/fig/kitti07_mesh.png" alt="kitti07_mesh" width="60%" /> </div>If you can not see the mesh, check that the Rviz plugin is sourced correctly. When mesh_visualization is disabled, only vertices are published as a point cloud.
3.2 Run Mai City dataset:
The dataset is available at [Mai Cit
