<div align="center">⚡FlashRAG: A Python Toolkit for Efficient RAG Research<div>

[ English | 中文 ]

<div align="center"> <a href="https://arxiv.org/abs/2405.13576" target="_blank"><img src=https://img.shields.io/badge/arXiv-b5212f.svg?logo=arxiv></a> <a href="https://huggingface.co/datasets/RUC-NLPIR/FlashRAG_datasets/" target="_blank"><img src=https://img.shields.io/badge/%F0%9F%A4%97%20HuggingFace%20Datasets-27b3b4.svg></a> <a href="https://www.modelscope.cn/datasets/hhjinjiajie/FlashRAG_Dataset" target="_blank"><img src=https://custom-icon-badges.demolab.com/badge/ModelScope%20Datasets-624aff?style=flat&logo=modelscope&logoColor=white></a> <a href="https://deepwiki.com/RUC-NLPIR/FlashRAG"><img src="https://devin.ai/assets/deepwiki-badge.png" alt="DeepWiki Document" height="20"/></a> <a href="https://github.com/RUC-NLPIR/FlashRAG/blob/main/LICENSE"><img alt="License" src="https://img.shields.io/badge/LICENSE-MIT-green"></a> <a><img alt="Static Badge" src="https://img.shields.io/badge/made_with-Python-blue"></a> </div> <h4 align="center"> <a href="#wrench-installation">Installation</a> | <a href="#sparkles-features">Features</a> | <a href="#rocket-quick-start">Quick-Start</a> | <a href="#gear-components"> Components</a> | <a href="#art-flashrag-ui"> FlashRAG-UI</a> | <a href="#robot-supporting-methods"> Supporting Methods</a> | <a href="#notebook-supporting-datasets--document-corpus"> Supporting Datasets</a> | <a href="#raised_hands-additional-faqs"> FAQs</a> </h4>

FlashRAG is a Python toolkit for the reproduction and development of Retrieval Augmented Generation (RAG) research. Our toolkit includes 36 pre-processed benchmark RAG datasets and 23 state-of-the-art RAG algorithms, including 7 reasoning-based methods that combine reasoning ability with retrieval.

With FlashRAG and provided resources, you can effortlessly reproduce existing SOTA works in the RAG domain or implement your custom RAG processes and components. Besides, we provide an easy-to-use UI:

https://github.com/user-attachments/assets/8ca00873-5df2-48a7-b853-89e7b18bc6e9

:sparkles: Features

Extensive and Customizable Framework: Includes essential components for RAG scenarios such as retrievers, rerankers, generators, and compressors, allowing for flexible assembly of complex pipelines.
Comprehensive Benchmark Datasets: A collection of 36 pre-processed RAG benchmark datasets to test and validate RAG models' performances.
Pre-implemented Advanced RAG Algorithms: Features 23 advancing RAG algorithms with reported results, based on our framework. Easily reproducing results under different settings.
🚀 Reasoning-based Methods: NEW! We now support 7 reasoning-based methods that combine reasoning ability with retrieval, achieving superior performance on complex multi-hop tasks.
Efficient Preprocessing Stage: Simplifies the RAG workflow preparation by providing various scripts like corpus processing for retrieval, retrieval index building, and pre-retrieval of documents.
Optimized Execution: The library's efficiency is enhanced with tools like vLLM, FastChat for LLM inference acceleration, and Faiss for vector index management.
Easy to Use UI : We have developed a very easy to use UI to easily and quickly configure and experience the RAG baselines we have implemented, as well as run evaluation scripts on a visual interface.

:mag_right: Roadmap

FlashRAG is still under development and there are many issues and room for improvement. We will continue to update. And we also sincerely welcome contributions on this open-source toolkit.

[x] Support OpenAI models
[x] Provdide instructions for each component
[x] Integrate sentence Transformers
[x] Support multimodal RAG
[x] Support reasoning-based methods
[ ] Inlcude more RAG approaches
[ ] Enhance code adaptability and readability
[ ] Add support for api-based retriever (vllm server)

:page_with_curl: Changelog

[25/11/06] 🎯 NEW Retriever! We have integrated a Web Search Engine-based Retriever, which seamlessly integrates with existing methods and can be enabled quickly with just a Serper API key! This enhancement significantly expands retrieval coverage and real-time capability, supporting dynamic information access and external knowledge augmentation. Experience a more flexible and powerful retrieval workflow now!

[25/08/06] 🎯 NEW! We have added support for Reasoning Pipeline, which is a new paradigm that combines reasoning ability and retrieval, representing work that includes R1-Searcher, Search-R1,.... We evaluate the performance of the pipeline on various RAG benchmarks, it can achieve F1 scores close to 60 on multi hop inference datasets such as HotpotQA. See it in result table.

[25/03/21] 🚀 Major Update! We have expanded our toolkit to support 23 state-of-the-art RAG algorithms, including 7 reasoning-based methods that significantly improve performance on complex reasoning tasks. This represents a major milestone in our toolkit's evolution!

[25/02/24] 🔥🔥🔥 We have added support for multimodal RAG, including MLLMs like Llava, Qwen, InternVL, and various multimodal retrievers with Clip architecture. More information can be found in our new version of arxiv article and our documentation. Try it!

[25/01/21] Our technical paper FlashRAG: A Python Toolkit for Efficient RAG Research is honored to have been accepted to the Resource Track of the 2025 ACM Web Conference (WWW 2025). Please Check it out!

[25/01/12] Introduce FlashRAG-UI, an easy to use interface. You can easily and quickly configure and experience the supported RAG methods and evaluate them on the benchmarks.

[25/01/11] We have added support for a new method RQRAG method, see it in reproduce_experiment.

[25/01/07] We have currently support the aggregation of multiple retrievers, see it in multi retriever usage.

[25/01/07] We have integrated a very flexible and lightweight corpus chunking library Chunkie, which supports various custom chunking methods (tokens, sentences, semantic, etc.). Use it in chunking doc corpus.

[24/10/21] We have released a version based on the Paddle framework that supports Chinese hardware platforms. Please refer to FlashRAG Paddle for details.

[24/10/13] A new in-domain dataset and corpus - DomainRAG have been added to the dataset. The dataset is based on the internal enrollment data of Renmin University of China, covering seven types of tasks, which can be used for conducting domain-specific RAG testing.

[24/09/24] We have released a version based on the MindSpore framework that supports Chinese hardware platforms. Please refer to FlashRAG MindSpore for details.

[24/09/18] Due to the complexity and limitations of installing Pyserini in certain environments, we have introduced a lightweight BM25s package as an alternative (faster and easier to use). The retriever based on Pyserini will be deprecated in future versions. To use retriever with bm25s, just set bm25_backend to bm25s in config.

[24/09/09] We add support for a new method Adaptive-RAG, which can automatically select the RAG process to execute based on the type of query. See it result in result table.

[24/08/02] We add support for a new method Spring, significantly improve the performance of LLM by adding only a few token embeddings. See it result in result table.

[24/07/17] Due to some unknown issues with HuggingFace, our original dataset link has been invalid. We have updated it. Please check the new link if you encounter any problems.

[24/07/06] We add support for a new method: Trace, which refine text by constructing a knowledge graph. See it results and details.

[24/06/19] We add support for a new method: IRCoT, and update the [result table](#robot-

FlashRAG

Install / Use

README

<div align="center">⚡FlashRAG: A Python Toolkit for Efficient RAG Research<div>

:link: Navigation

:sparkles: Features

:mag_right: Roadmap

:page_with_curl: Changelog