RAI

RAI is a flexible AI agent framework to develop and deploy Embodied AI features for your robots.

📚 Visit robotecai.github.io/rai for the latest documentation, setup guide and tutorials. 📚

---

<div align="center">

---

</div>

---

🎯 Overview

| Category | Description | Features | | ------------------------------ | ------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | 🤖 Multi-Agent Systems | Empowering robotics with advanced AI capabilities | • Seamlessly integrate Gen AI capabilities into your robots<br>• Enable sophisticated agent-based architectures | | 🔄 Robot Intelligence | Enhancing robotic systems with smart features | • Add natural human-robot interaction capabilities<br>• Bring flexible problem-solving to your existing stack<br>• Provide ready-to-use AI features out of the box | | 🌟 Multi-Modal Interaction | Supporting diverse interaction capabilities | • Handle diverse data types natively<br>• Enable rich sensory integration<br>• Process multiple input/output modalities simultaneously |

RAI framework

• [x] rai core: Core functionality for multi-agent system, human-robot interaction and multi-modalities.
• [x] rai whoami: Tool to extract and synthesize robot embodiment information from a structured directory of documentation, images, and URDFs.
• [x] rai_asr: Speech-to-text models and tools.
• [x] rai_tts: Text-to-speech models and tools.
• [x] rai_sim: Package for connecting RAI to simulation environments.
• [x] rai_bench: Benchmarking suite for RAI. Test agents, models, tools, simulators, etc.
• [x] rai_perception: Object detection tools based on open-set models and machine learning techniques.
• [x] rai_nomad: Integration with NoMaD for navigation.
• [ ] rai_finetune: Finetune LLMs on your embodied data.

Getting started

See Quick setup guide.

Simulation demos

Try RAI yourself with these demos: | Application | Robot | Description | Docs Link | | ------------------------------------------ | ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------- | | Mission and obstacle reasoning in orchards | Autonomous tractor | In a beautiful scene of a virtual orchard, RAI goes beyond obstacle detection to analyze best course of action for a given unexpected situation. | link | | Manipulation tasks with natural language | Robot Arm (Franka Panda) | Complete flexible manipulation tasks thanks to RAI and Grounded SAM 2 | link | | Autonomous mobile robot demo | Husarion ROSbot XL | Demonstrate RAI's interaction with an autonomous mobile robot platform for navigation and control | link | | Agentic mobile manipulator | RB-KAIROS | A comprehensive demo with on-board execution | link |

Community

Embodied AI Community Group

RAI is one of the main projects in focus of the Embodied AI Community Group. If you would like to join the next meeting, look for it in the ROS Community Calendar.

Publicity

• A talk about RAI at ROSCon 2024.

RAI Q&A

Please take a look at Q&A.

Developer Resources

See our documentation for a deeper dive into RAI, including instructions on creating a configuration specifically for your robot.

Contributing

You are welcome to contribute to RAI! Please see our Contribution Guide.

Citation

If you find our work helpful for your research, please consider citing the following BibTeX entry.

@misc{rachwał2025raiflexibleagentframework,
      title={RAI: Flexible Agent Framework for Embodied AI},
      author={Kajetan Rachwał and Maciej Majek and Bartłomiej Boczek and Kacper Dąbrowski and Paweł Liberadzki and Adam Dąbrowski and Maria Ganzha},
      year={2025},
      eprint={2505.07532},
      archivePrefix={arXiv},
      primaryClass={cs.MA},
      url={https://arxiv.org/abs/2505.07532},
}