COVINS-(G) -- A (Generic) Framework for Collaborative Visual-Inertial SLAM and Multi-Agent 3D Mapping

Version 2.0 (Support for COVINS-G: A Generic Back-end for Collaborative Visual-Inertial SLAM)

COVINS is an accurate, scalable, and versatile visual-inertial collaborative SLAM system, that enables a group of agents to simultaneously co-localize and jointly map an environment. COVINS provides a server back-end for collaborative SLAM, running on a local machine or a remote cloud instance, generating collaborative estimates from map data contributed by different agents running Visual-Inertial Odomety (VIO) and sharing their map with the back-end.

With the COVINS-G release, we make the server back-end flexible enabling compatibility with any arbitrary VIO/Stereo front-end, including, for example, off-the-shelf cameras with odometry capabilities, such as the Realsense T265.

We provide guidance and examples how to run COVINS and COVINS-G on the EuRoC dataset, as well as information beyond basic deployment, for example how the COVINS back-end can be deployed on a remote cloud computing instance. Instructions on running COVINS-G with different cameras such as Intel Realsense D455 and T265 Tracking camera, as well as different frontends like VINS-Fusion, ORB-SLAM3 and SVO-pro are also provided.

Index

• 1 Related Publications
• 2 License
• 3 Basic Setup
  • Environment Setup
  • COVINS Installation
  • Installing ROS Support for the ORB-SLAM3 Front-End
  • Installing VINS-Fusion Front-End
• 4 Running COVINS
  • COVINS Back-End
    • Running the COVINS Server Back-End
    • Running the ORB-SLAM3 Front-End
    • Visualization
    • User Interaction
    • Parameters
    • Output Files
    • Running COVINS with ROS
  • COVINS-G Back-End
    • Running the COVINS-G Server Back-End
    • ORB-SLAM3 Front-End
    • VINS-Fusion Front-End
    • ROS-based Front-End Wrapper
    • Mixture of Front-Ends
    • Realsense T265 Tracking Camera
    • Custom Dataset (Realsense D455)
    • Using SIFT Features
    • Visualization
    • User Interaction
    • Parameters
    • Output Files
• 5 Docker Implementation
  • Building the Docker Image
  • Running the Docker Image
• 6 Extended Functionalities
  • Deployment of the COVINS Back-End in an AWS Cloud Instance
  • Interfacing a Custom VIO System with COVINS
  • Map Re-Use Onboard the Agent (Only for COVINS)
• 7 Limitations and Known Issues

<a name="related_publications"></a>

1 Related Publications

[COVINS-G] Manthan Patel, Marco Karrer, Philipp Bänninger and Margarita Chli. COVINS-G: A Generic Back-end for Collaborative Visual-Inertial SLAM. IEEE International Conference on Robotics and Automation (ICRA), 2023. PDF

[COVINS] Patrik Schmuck, Thomas Ziegler, Marco Karrer, Jonathan Perraudin and Margarita Chli. COVINS: Visual-Inertial SLAM for Centralized Collaboration. IEEE International Symposium on Mixed and Augmented Reality (ISMAR), 2021. PDF

[Redundancy Detection] Patrik Schmuck and Margarita Chli. On the Redundancy Detection in Keyframe-based SLAM. IEEE International Conference on 3D Vision (3DV), 2019. PDF.

[System Architecture] Patrik Schmuck and Margarita Chli. CCM‐SLAM: Robust and Efficient Centralized Collaborative Monocular Simultaneous Localization and Mapping for Robotic Teams. Journal of Field Robotics (JFR), 2019. PDF

[Collaborative VI-SLAM] Patrik Schmuck, Marco Karrer and Margarita Chli. CVI-SLAM - Collaborative Visual-Inertial SLAM. IEEE Robotics and Automation Letters (RA-L), 2018. PDF

Video:

COVINS-G: | COVINS: --- | --- <a href="https://www.youtube.com/embed/FoJfXCfaYDw" target="_blank"><img src=".aux/COVINS-G_cover.png" alt="Mesh" width="420" height="270" border="10" /></a> | <a href="https://www.youtube.com/embed/FxJTY5x1fGE" target="_blank"><img src="https://i.ytimg.com/vi/FxJTY5x1fGE/hqdefault.jpg" alt="Mesh" width="360" height="270" border="10" /></a>

<a name="license"></a>

2 License

COVINS is released under a GPLv3 license. For a list of code/library dependencies (and associated licenses), please see thirdparty_code.md.

For license-related questions, please contact the authors: collaborative (dot) slam (at) gmail (dot) com.

If you use COVINS in an academic work, please cite:

@article{schmuck2021covins,
  title={COVINS: Visual-Inertial SLAM for Centralized Collaboration},
  author={Schmuck, Patrik and Ziegler, Thomas and Karrer, Marco and Perraudin, Jonathan and Chli, Margarita},
  journal={arXiv preprint arXiv:2108.05756},
  year={2021}
}

If you use the COVINS-G extension in an academic work, please cite both the original COVINS publication and additionally:

@article{patel23covinsg,
  title = {COVINS-G: A Generic Back-end for Collaborative Visual-Inertial SLAM},
  doi = {10.48550/ARXIV.2301.07147},
  url = {https://arxiv.org/abs/2301.07147},
  author = {Patel, Manthan and Karrer, Marco and Bänninger, Philipp and Chli, Margarita},
  publisher = {arXiv},
  year = {2023},
}

<a name="setup"></a>

3 Basic Setup

This section explains how you can build the COVINS server back-end, as well as the provided version of the ORB-SLAM3 front-end able to communicate with the back-end. Morover, the COVINS-G extension provides a generic ROS front-end wrapper to support any arbitrary VIO/tracking cameras as well as support for VINS-Fusion. We provide installation instructions for Ubuntu 18.04 as well as 20.04. Note that we also provide a Docker implementation for simplified deployment of COVINS.

Note: Please pay attention to the CMAKE_BUILD_TYPE. Particularly, building parts of the code with march=native can cause problems on some machines.

<a name="setup_env"></a>

Environment Setup

Dependencies

• sudo apt-get update
• Install dependencies: sudo apt-get install libpthread-stubs0-dev build-essential cmake git doxygen libsuitesparse-dev libyaml-cpp-dev libvtk6-dev python3-wstool libomp-dev libglew-dev
• catkin_tools (from the catkin_tools manual)
  • sudo sh -c 'echo "deb http://packages.ros.org/ros/ubuntu `lsb_release -sc` main" > /etc/apt/sources.list.d/ros-latest.list'
  • wget http://packages.ros.org/ros.key -O - | sudo apt-key add -
  • sudo apt-get update
  • sudo apt-get install python3-catkin-tools
• ROS
  • Melodic (Ubuntu 18)
  • Noetic (Ubuntu 20)

Set up your workspace

This will create a workspace for COVINS as ~/ws/covins_ws. All further commands will use this path structure - if you decide to change the workspace path, you will need to adjust the commands accordingly.

• cd ~
• mkdir -p ws/covins_ws/src
• cd ~/ws/covins_ws
• catkin init
• ROS Setup
  • U18/Melodic: catkin config --extend /opt/ros/melodic/
  • U20/Noetic: catkin config --extend /opt/ros/noetic/
• catkin config --merge-devel
• catkin config --cmake-args -DCMAKE_BUILD_TYPE=RelWithDebInfo

<a name="setup_covins"></a>

COVINS Installation

Note on Virtual Machines

If you are installing COVINS in a Virtual Machine (VM) or a computer with fewer resources, make sure to allocate enough memory and/or swap to ensure a successful installation. We have successfully tested the COVINS installation with a VM with 16 GB RAM and 10 GB swap. Instructions how to change the swap size can be found here.

Installation Manual

We provide a script (covins/install_file.sh) that will perform a full installation of COVINS, including back-end, front-end, and third-party packages, if the environment is set up correctly. If the installation fails, we strongly recommend executing the steps in the build script manually one by one. The script might not perform a correct installation under certain circumstance