Robotized fulfillment center

An Open 3D Engine (O3DE) multi-robot demo.

The project

This project demonstrates O3DE use for a complex robotic simulation through integration with modern ROS 2 stacks: nav2 and MoveIt2. You can learn more about the features of O3DE for robotics and how to get started in O3DE documentation.

Levels

DemoLevel1: 30x100 meters scene with 4 conveyor belts, and 4 robotic arms. Suitable for 4-8 AMRs.
DemoLevel2: 90x100 meters, three times larger, and 12 robotic arms, suitable for 12-24 AMRs.
RobotsSuperShot: a level showcasing 3D models, with several robots, a human, and a forklift. Some robots are not equipped with all components yet, but you are welcome to try and make them work!
RobotImportLevel: a small enclosed space with a table, good for importing your own robot.
DemoStereo: a level showcasing HIL setup with AMD KRIA and stereo cameras. Visit KRIA depth demo for more information.

Detailed description

UR20 robot arms controlled by MoveIt2 with pilz_industrial_motion_planner. Boxes are supplied by conveyor belts, which are implemented through spawning when below a certain number in an area. UR20 arms are placing boxes based on a ground truth vision system, which means they look at the scene and that there is no error in pose measurement. UR20 arms start working as soon as an immobile pallet is detected in their load area and will load a configurable number of boxes (up to 18, by default 18) on each pallet.

Pallets are moved around by robots modeled after OTTO 600. Note that these AMRs do not use the software of real OTTO 600 and do not have the same sensors. They can navigate thanks to front/back lidar sensors and operate cargo lifts. OTTO 600 robots have assigned task loops, which are loading, wrapping, and delivering cargo to the other end of the warehouse. They use a nav2 action server to realize their paths, and also a custom path follow solution for docking and unloading (for simplicity). Note that robots follow their task independently but see and avoid each other and people walking around. On the other hand, human workers don't see robots, as they use navigation through a Gem and only consider static scene objects.

How does it look like

The project includes

Scenery initially created using a Warehouse project template, but many new models have been added.
Robotic Arms imported using URDF description provided by Universal Robotics for their UR20 collaborative robot. The prefabs are available in a separate Gem.
AMRs instantiated prefabs of OTTO600 and OTTO1500 robots from OTTO Motors.
Boxes that are transported using conveyor belts from Warehouse Automation Gem and palletized.

Requirements

Platforms

The project runs on Ubuntu 22.04 with ROS 2 Humble or ROS 2 Iron.

💡 Note: This demo is not supported on ROS 2 Jazzy! 💡 Note: This demo is not supported on Windows!

Hardware

The demo is rather demanding, as it aims to show what is possible. Minimum specs are not determined, but we ran it on:

AMD Radeon RX 6800 / NVIDIA GeForce RTX 3070 GPU (8 GB).
AMD Ryzen 9 5900HX CPU / Intel i7-11800H.
64 GB RAM.

For more FPS, a larger scene, and more robots, we used:

AMD Radeon RX 7900 XT / NVIDIA RTX 3080 Ti (or better) GPU (16 GB).
AMD Ryzen 9 7950X / Intel i7-12900KF (24 cores) CPU.
64 GB RAM.

Docker container environment

Follow the instructions in ./Docker/README file to build and run the project using docker virtualization.

Project Setup

ROS 2 middleware

This project should be used with the rmw_cyclonedds_cpp as the ROS 2 middleware. MoveIt2 does not recommend usage of the default RMW and as it is a part of this project using the default RMW will not work.

Install the CycloneDDS RMW by installing its package:

sudo apt install ros-${ROS_DISTRO}-rmw-cyclonedds-cpp

After the installation add this command to your .bashrc or equivalent file.

export RMW_IMPLEMENTATION=rmw_cyclonedds_cpp

This will change the default RMW implementation to CycloneDDS. Source your new configuration:

source ~/.bashrc

Note: The ROS2 daemon may need to be restarted to use the CycloneDDS RMW. Use ros2 daemon stop and ros2 daemon start to restart the daemon.

O3DE

Refer to the O3DE System Requirements documentation to make sure that the system/hardware requirements are met.
Please follow the instructions to set up O3DE from GitHub.
This project was tested on O3DE 2505.1. o3de 2505.1 and o3de-extras 2505.1 are recommended versions, but the newer point-releases should work.

The following commands should prepare O3DE (assuming that the project repository is cloned into ${RC2023_WORKDIR}):

First, initialize the git submodules (this fetches engine/o3de, engine/o3de-extras, and the Gems in gems/):

cd ${RC2023_WORKDIR}
git submodule update --init --recursive
git submodule foreach 'git lfs install && git lfs pull'

Then register the engine:

cd ${RC2023_WORKDIR}/engine/o3de
python/get_python.sh
scripts/o3de.sh register --this-engine

ROS 2 Gem and other Gems

This project uses the following Gems:

Please make sure to follow the installation guide in the Project Configuration file up until the creation of a new Project.

To learn more about how the Gem works check out the Concepts and Structures.

Note that the Gem instructions include the installation of ROS 2 with some additional packages.

All Gems are included as git submodules and are fetched by the git submodule update command above.

The Gems are open to your contributions!

ROS 2 packages

Make sure to install the necessary ROS 2 packages.

sudo apt install ros-${ROS_DISTRO}-ackermann-msgs ros-${ROS_DISTRO}-control-toolbox ros-${ROS_DISTRO}-nav-msgs ros-${ROS_DISTRO}-gazebo-msgs ros-${ROS_DISTRO}-vision-msgs ros-${ROS_DISTRO}-ur-msgs ros-${ROS_DISTRO}-moveit-servo ros-${ROS_DISTRO}-moveit-visual-tools ros-${ROS_DISTRO}-moveit ros-${ROS_DISTRO}-pilz-industrial-motion-planner ros-${ROS_DISTRO}-controller-manager ros-${ROS_DISTRO}-ur-client-library ros-${ROS_DISTRO}-nav2-common ros-${ROS_DISTRO}-navigation2 libopencv-dev ros-${ROS_DISTRO}-nav2-map-server

Project

You need to build and source the ROS 2 workspace first as it contains custom messages that the simulator also uses. This workspace depends on submodules that need to be pulled first. This is done through the script (setup_submodules.bash) that selects a submodule's version based on the detected ROS 2 distribution.

cd ${RC2023_WORKDIR}/ROSCon2023Demo/ros2_ws
./setup_submodules.bash

Now install all dependencies of submodules.

sudo apt install python3-colcon-common-extensions python3-vcstool python3-rosdep2
rosdep update
rosdep install --ignore-src --from-paths src/Universal_Robots_ROS2_Driver -y
# disable not needed packages - we need bringup and description package
touch src/Universal_Robots_ROS2_Driver/ur_controllers/COLCON_IGNORE
touch src/Universal_Robots_ROS2_Driver/ur_robot_driver/COLCON_IGNORE
touch src/Universal_Robots_ROS2_Driver/ur_calibration/COLCON_IGNORE

Then build and source the workspace.

cd ${RC2023_WORKDIR}/ROSCon2023Demo/ros2_ws
colcon build --symlink-install
source install/setup.bash # adjust to your shell

The source command needs to be done in the same console where you build and run O3DE.

Make sure the following tools and libraries are installed on your system (they are required to build the project):

sudo apt install ninja-build libunwind-dev libxcb-xkb-dev libxcb-xfixes0-dev libxkbcommon-x11-dev libxcb-xinput-dev

Now, assuming that the project's repo was cloned to ${RC2023_WORKDIR}:

cd ${RC2023_WORKDIR}/ROSCon2023Demo/Project
cmake -B build/linux -G "Ninja Multi-Config" -DLY_DISABLE_TEST_MODULES=ON -DLY_STRIP_DEBUG_SYMBOLS=ON
cmake --build build/linux --config profile --target Edito

ROSCon2023Demo

Install / Use

README