RMSearch

Reward-Model Search (RMSearch) toolkit that scores 'keys' (documents, agents, tools, steps) from 'queries' (questions, context) with graph acceleration.

Generate Convert Improve

Install / Use

/learn @kyotoai/RMSearch

About this skill

Quality Score

0/100

README

[![Contributors][contributors-shield]][contributors-url] [![Forks][forks-shield]][forks-url] [![Stargazers][stars-shield]][stars-url] [![Issues][issues-shield]][issues-url] [![MIT License][license-shield]][license-url] [![LinkedIn][linkedin-shield]][linkedin-url]

<br /> <div align="center"> <a href="https://kyotoai.org"> <img src="images/RMSearch_fig.png" alt="Logo" width="640" height="360"> </a> <h3 align="center">RMSearch</h3> <p align="center"> RMSearch is a high intellectual search tool using reward model instead of semantic embedding model. Agentic search is a good application and RMSearch enables step by step CoT reasoning and optimize the reasoning path! <br /> <a href="https://github.com/kyotoai/RMSearch/tree/main/demo"><strong>Explore the docs »</strong></a> <br /> <br /> <a href="https://github.com/kyotoai/RMSearch/tree/main/demo">View Demo</a> · <a href="https://github.com/kyotoai/RMSearch/issues/new?labels=bug&template=bug-report---.md">Report Bug</a> · <a href="https://github.com/kyotoai/RMSearch/issues/new?labels=enhancement&template=feature-request---.md">Request Feature</a> </p> </div>  <details> <summary>Table of Contents</summary> <ol> <li> <a href="#about-the-project">About The Project</a> <ul> <li><a href="#built-with">Built With</a></li> </ul> </li> <li> <a href="#getting-started">Getting Started</a> <ul> <li><a href="#prerequisites">Prerequisites</a></li> <li><a href="#installation">Installation</a></li> </ul> </li> <li><a href="#usage">Usage</a></li> <li><a href="#roadmap">Roadmap</a></li> <li><a href="#contributing">Contributing</a></li> <li><a href="#license">License</a></li> <li><a href="#contact">Contact</a></li> <li><a href="#acknowledgments">Acknowledgments</a></li> </ol> </details>

About The Project

Search The Best Agent To Make Deep Reasoning

<br /> <div align="center"> <img src="images/SEIMEI_example.png" alt="seimei" width="640" height="360"> </div> <br /> Here's the example of how SEIMEI works. Each agent interacts with LLM and document and makes inference. These inferences are automatically integrated by search engine and gives an answer of question. <br /> <div align="center"> <img src="images/SEIMEI_train_example.png" alt="seimei" width="640" height="360"> </div> <br /> By training search engine, we can optimize the thinking steps like o1 or deepseek-r1!! <p align="right">(<a href="#readme-top">back to top</a>)</p>

The Most Intelligent Search Engine

Our proprietary search model performs better than semantic embedding model(so called vector search). The graph above shows the result of training our model (3b) and e5-mistral-7b model to search best agents. While the vector search model cannot really retrieve best agents(because problems and agents do not have similar sentences), our proprietary search model can learn what agents are needed to solve a question and retrive the best ones!!

<a href="https://github.com/kyotoai/SEIMEI/tree/main/demo"><strong>See more details »</strong></a>

Achives State Of The Art Result

We acheved an improvement of bigcodebench/deepseek-r1 by our search engine!!

<a href="https://github.com/kyotoai/SEIMEI/tree/main/demo"><strong>See more details »</strong></a>

Applications of SEIMEI

SEIMEI can be applied to make these useful functions!!

<a href="https://github.com/kyotoai/SEIMEI/tree/main/demo"><strong>See more details »</strong></a>

Built With

[![vLLM][vllm.ai]][vllm-url]
[![Hugging Face][huggingface.co]][huggingface-url]

<p align="right">(<a href="#readme-top">back to top</a>)</p>

Getting Started

This is an example of how you build RMSearch on local gpu or cloud server gpu. You can use it by downloading this directory into your local folder.

Prerequisites

This library requires cuda+torch environment with GPU. The memory of GPU should be higher than 12GB to run the sample.

Installation

by pip install

Install seimei (not prepared yet)
```
pip install rmsearch
```

by downloading SEIMEI repository

Download the repo

git clone https://github.com/kyotoai/RMSearch.git
cd RMSearch
pip install .

Usage

We are still developing this library. Please wait for it completing soon!

Quick Start

Define search instance

from rmsearch import Search
search = Search(model_name = "/workspace/llama3b-rm",
        tensor_parallel_size = 1,
        pipeline_parallel_size = 1,)

Search the most relevant keys

queries = ["How to make LLM?", "What's the capital of Japan?"] * 5
keys = ["LLM is Large Language Model which can be made ..."*7, "Japanese capital is ..."*7] * 5
output = await search(queries, keys)
print(output)

Train Reward Model For Your Own Tasks (For Developpers)

See the full code in /examples/example_train2.ipynb

Generate Tree Structural LLM Output

# This is an example of making agentic tree to optimize brainstorming
num_depth = 2
num_branch = 2
total_num_node = num_branch**(num_depth+1) - 2

bs_agent_tree = {}  

# Recursive function to make agentic tree
def build_bs_agent_tree(**kwargs):

    # make kwargs_ from kwargs

    def _grow(node, depth, **kwargs_):
        if depth == num_depth:
            return

        for b in range(num_branch):
            child = {
                "agent"   : "something",
                "node_ids" : node["node_ids"] + [b],
            }
            node["children"].append(child)

            _grow(child, depth + 1, **kwargs_)

    root = {"agent": "root", "node_ids": [], "children": []}
    _grow(root, 0, **kwargs_)

    return root

# Get output from LLM
def get_assistant_msg(node, **llm_kwargs):
    output = get_output(node["agent"], **llm_kwargs)  # LLM function
    return output

# Walk tree to add output
def populate_tree(node, **llm_kwargs):
    if node["agent"] is not "root":         # skip dummy root
        node["output"] = get_assistant_msg(
            node,
            **llm_kwargs,
        )

    for child in node["children"]:
        populate_tree(child, **llm_kwargs)

def check(output):
    # some function to check its novelty
    return evaluation

# Walk tree to add evaluation towards each node's output
def add_eval_to_tree(node, depth):

    if not node["output"] != None:
        if node["eval"] == []:
            node["eval"] = check(node["output"])

    for child in node["children"]:
        add_eval_to_tree(child, depth + 1)
        

# Asign agents to each node in bs_agent_tree
bs_agent_tree = build_bs_agent_tree(agents, num_depth=num_depth, num_branch=num_branch)  # bs_agent_tree : [{"agent":"root", "node_id":[0], "children":[{"agent":"agent1", "node_id":[0, 0], "children":[...]}, ...]

# Get llm output in each node
populate_tree(bs_agent_tree, **kwargs)  # bs_agent_tree = [{"agent":"root", "node_id":[0], "children":[{"agent":"agent1", "output":"...", "node_id":[0, 0], "children":[...]}, ...]

# Get evaluation of output in each node
add_eval_to_tree(bs_agent_tree, 0)  # bs_agent_tree = [{"agent":"root", "node_id":[0], "children":[{"agent":"agent1", "output":"...", "eval":"...", "node_id":[0, 0], "children":[...]}, ...]

Make dataset_list

# dataset_list should be 
# [{"query":str, "chosen_key":str, "rejecte

Related Skills

node-connect

341.2k

Diagnose OpenClaw node connection and pairing failures for Android, iOS, and macOS companion apps

frontend-design

84.5k

Create distinctive, production-grade frontend interfaces with high design quality. Use this skill when the user asks to build web components, pages, or applications. Generates creative, polished code that avoids generic AI aesthetics.

openai-whisper-api

341.2k

Transcribe audio via OpenAI Audio Transcriptions API (Whisper).

commit-push-pr

84.5k

Commit, push, and open a PR