Multi AI Agent Systems <br>using OpenAI's Assistants API (Experts.js)

Experts.js is the easiest way to create and deploy OpenAI's Assistants and link them together as Tools to create a Panel of Experts system with expanded memory and attention to detail.

Made via support ❤️ by Custom Ink | Tech

Overview

The new Assistants API from OpenAI sets a new industry standard, significantly advancing beyond the widely adopted Chat Completions API. It represents a major leap in the usability of AI agents and the way engineers interact with LLMs. Paired with the cutting-edge GPT-4o mini model, Assistants can now reference attached files & images as knowledge sources within a managed context window called a Thread. Unlike Custom GPTs, Assistants support instructions up to 256,000 characters, integrate with 128 tools, and utilize the innovative Vector Store API for efficient file search on up to 10,000 files per assistant.

Experts.js aims to simplify the usage of this new API by removing the complexity of managing Run objects and allowing Assistants to be linked together as Tools.

import { Assistant, Thread } from "experts";

const thread = await Thread.create();
const assistant = await Assistant.create();
const output = await assistant.ask("Say hello.", thread.id);
console.log(output) // Hello

More importantly, Experts.js introduces Assistants as Tools, enabling the creation of Multi AI Agent Systems. Each Tool is an LLM-backed Assistant that can take on specialized roles or fulfill complex tasks on behalf of their parent Assistant or Tool. Allowing for complex orchestration workflows or choreographing a series of tightly knit tasks. Shown here is an example of a company assistant with a product catalog tool which itself has a LLM backed tool to create OpenSearch queries.

Installation

Install via npm. Usage is very simple, there are only three objects to import.

npm install experts

Experts.js supports both ES6 import syntax and CommonJS require statements.

import { Assistant, Tool, Thread } from "experts";

• Assistants - The main object that represents an AI agent.
• Tools - An Assistant that can be used by other Assistants.
• Threads - A managed context window for your agents.

Assistants

The constructor of our Assistant facade object requires a name, description, and instructions. The third argument is a set of options which directly maps to all the request body options outlined in the create assistant documentation. All examples in Experts.js are written in ES6 classes for simplicity. The default model is gpt-4o-mini.

class MyAssistant extends Assistant {
  constructor() {
    super({
      name: "My Assistant",
      instructions: "...",
      model: "gpt-4o-mini",
      tools: [{ type: "file_search" }],
      temperature: 0.1,
      tool_resources: {
        file_search: {
          vector_store_ids: [process.env.VECTOR_STORE_ID],
        },
      },
    });
  }
}

const assistant = await MyAssistant.create();

The Experts.js async Assistant.create() base factory function is a simple way to create an assistant using the same constructor options.

const assistant = Assistant.create({
  name: "My Assistant",
  instructions: "...",
  model: "gpt-4o-mini",
});

[!IMPORTANT]
Creating assistants without an id parameter will always create a new assistant. See our deployment section for more information.

Simple Ask Interface

The ask() function is a simple interface to ask or instruct your assistant(s). It requires a message and a thread identifier. More on Threads below. The message can be a string or native OpenAI message object. This is where Experts.js really shines. You never have to manage Run objects or their Run Steps directly.

const output = await assistant.ask("...", threadID)
const output = await assistant.ask({ role: "user", content: "..." }, threadID);

Adding Tools

Normal OpenAI tools and function calling are supported via our constructors options object via tools and tool_resources. Experts.js also supports adding Assistants as Tools. More information on using Assistants as Tools can be found in the next section. Use the addAssistantTool function to add an Assistant as a Tool. This must happen after super() in your Assistant's constructor.

class MainAssistant extends Assistant {
  constructor() {
    super({
      name: "Company Assistant",
      instructions: "...",
    });
    this.addAssistantTool(ProductsTools);
  }
}

Streaming & Events

By default, Experts.js leverages the Assistants Streaming Events. These allow your applications to receive text, image, and tool outputs via OpenAI's server-send events. We leverage openai-node's stream helpers and surface these events along with a few custom ones giving your assistants to tap into the complete lifecycle of a Run.

const assistant = await MainAssistant.create();
assistant.on("textDelta", (delta, _snapshot) => {
  process.stdout.write(delta.value)
});

All openai-node streaming events are supported via our Assistant's on() function. The available event names are: event, textDelta, textDone, imageFileDone, toolCallDelta, runStepDone, toolCallDone, and end

[!IMPORTANT]
OpenAI's server-send events are not async/await friendly.

If your listeners need to perform work in an async fashion, such as redirecting tool outputs, consider using our extensions to these events. They are called in this order after the Run has been completed. The available async event names are: textDoneAsync, imageFileDoneAsync, runStepDoneAsync, toolCallDoneAsync, and endAsync.

Advanced Features

If you want to lazily standup additional resources when an assistant's create() function is called, implement the beforeInit() function in your class. This is an async method that will be called before the assistant is created.

async beforeInit() {
  await this.#createFileSearch();
}

Likewise, the afterInit() function can be used. For example, to write out newly created Assistants' IDs to an environment file.

async afterInit() {
  // ...
}

All Assistant events receive an extra Experts'js metadata argument. An object that contains the Run's stream. This allows you to use the openai-node's helper functions such as currentEvent, finalMessages, etc.

assistant.on("endAsync", async (metadata) => {
  await metadata.stream.finalMessages();
});

Tools

Using an Assistant as a Tool is central focal point of the Experts.js framework. Tools are a subclass of Assistant and encapsulate the interface for their parent objects. In this way Experts.js tools are reusable components in your agentic architecture. Our examples illustrate a basic message passing pattern, for brevity. You should leverage all of OpenAI's tool and function calling features to their fullest.

class EchoTool extends Tool {
  constructor() {
    super({
      name: "Echo Tool",
      instructions: "Echo the same text back to the user",
      parentsTools: [
        {
          type: "function",
          function: {
            name: "echo",
            description: description,
            parameters: {
              type: "object",
              properties: { message: { type: "string" } },
              required: ["message"],
            },
          },
        },
      ],
    });
  }
}

[!CAUTION] It is critical that your tool's function name be unique across its parent's entire set of tool names.

As such, Tool class names are important and help OpenAI's models decide which tool to call. So pick a good name for your tool class. For example, ProductsOpenSearchTool will be products_open_search and clearly helps the model infer along with the tool's description what role it performs.

Tools are added to your Assistant via the addAssistantTool function. This function will add the tool to the assistant's tools array and update the assistant's configuration. This must happen after super() in your Assistant's constructor.

class MainAssistant extends Assistant {
  constructor() {
    super({
      na