<img src="https://storage.googleapis.com/assistly/static/realchar/realchar.svg" height="24px" style="padding-top:4px"/>RealChar. - Your Realtime AI Character

<br/> <div align="center"> <img src="https://storage.googleapis.com/assistly/static/realchar/logo.png" alt="RealChar-logo" width="80%" style="padding: 40px"/> </div> <br/> <p align="center"> 🎙️🤖<em>Create, customize and talk to your AI Character/Companion in realtime</em>🎙️🤖 </p> <div align="center"> <a href="https://realchar.ai/join-discord"> <img src="https://img.shields.io/badge/discord-join%20chat-blue.svg?style=for-the-badge" alt="Join our Discord" height="20"> </a> <a href="https://twitter.com/agishaun"> <img alt="Twitter Follow" src="https://img.shields.io/twitter/follow/agishaun?style=for-the-badge" height="20"> <a href="https://github.com/Shaunwei/RealChar"> <img alt="GitHub" src="https://img.shields.io/github/stars/Shaunwei/RealChar?style=for-the-badge&color=gold" height="20"> </a> <a href="https://github.com/Shaunwei/RealChar/commits/main"> <img alt="GitHub" src="https://img.shields.io/github/last-commit/Shaunwei/RealChar/main?style=for-the-badge" height="20"> </a> <a href="https://github.com/Shaunwei/RealChar/blob/main/README.md" target="_blank"> <img src="https://img.shields.io/static/v1?label=license&message=MIT&color=green&style=for-the-badge" alt="License" height="20"> </a> <a href="https://hub.docker.com/repository/docker/shaunly/real_char/general" target="_blank"> <img alt="Docker Pulls" src="https://img.shields.io/docker/pulls/shaunly/real_char?style=for-the-badge" height="20"> </a> </div>

✨ Demo

Try our site at RealChar.ai

Not sure how to pronounce RealChar? Listen to this 👉 audip

Demo 1 - with Santa Claus!

https://github.com/Shaunwei/RealChar/assets/5101573/6b35a80e-5503-4850-973d-254039bd383c

Demo 2 - with AI Elon about cage fight!

https://github.com/Shaunwei/RealChar/assets/5101573/5de0b023-6cf3-4947-84cb-596f429d109e

Demo 3 - with AI Raiden about AI and "real" memory

https://github.com/Shaunwei/RealChar/assets/5101573/62a1f3d1-1166-4254-9119-97647be52c42

Demo settings: Web, GPT4, ElevenLabs with voice clone, Chroma, Google Speech to Text

🎯 Key Features

• Easy to use: No coding required to create your own AI character.
• Customizable: You can customize your AI character's personality, background, and even voice
• Realtime: Talk to or message your AI character in realtime
• Multi-Platform: You can talk to your AI character on web, terminal and mobile(Yes. we open source our mobile app)
• Most up-to-date AI: We use the most up-to-date AI technology to power your AI character, including OpenAI, Anthropic Claude 2, Chroma, Whisper, ElevenLabs, etc.
• Modular: You can easily swap out different modules to customize your flow. Less opinionated, more flexible. Great project to start your AI Engineering journey.

🔬 Tech stack

<div align="center"> <img src="https://storage.googleapis.com/assistly/static/realchar/techstackv004.jpg" alt="RealChar-tech-stack" width="100%" style="padding: 20px"/> </div>

• ✅Web: React JS, Vanilla JS, WebSockets
• ✅Mobile: Swift, WebSockets
• ✅Backend: FastAPI, SQLite, Docker
• ✅Data Ingestion: LlamaIndex, Chroma
• ✅LLM Orchestration: LangChain, Chroma
• ✅LLM: ReByte, OpenAI GPT3.5/4, Anthropic Claude 2, Anyscale Llama2
• ✅Speech to Text: Local WhisperX, Local Whisper, OpenAI Whisper API, Google Speech to Text
• ✅Text to Speech: ElevenLabs, Edge TTS, Google Text to Speech
• ✅Voice Clone: ElevenLabs

📚 Comparison with existing products

<div align="center"> <img src="https://storage.googleapis.com/assistly/static/realchar/compare.png"> </div>

📀 Quick Start - Installation via Docker

1. Create a new .env file

   cp .env.example .env
   

   Paste your API keys in .env file. A single ReByte or OpenAI API key is enough to get started.

   You can also configure other API keys if you have them.

2. Start the app with docker-compose.yaml

   docker compose up
   

   If you have issues with docker (especially on a non-Linux machine), please refer to https://docs.docker.com/get-docker/ (installation) and https://docs.docker.com/desktop/troubleshoot/overview/ (troubleshooting).

3. Open http://localhost:3000 and enjoy the app!

💿 Developers - Installation via Python

• Step 1. Clone the repo

  git clone https://github.com/Shaunwei/RealChar.git && cd RealChar
  

• Step 2. Install requirements

  Install portaudio and ffmpeg for audio

  # for mac
  brew install portaudio
  brew install ffmpeg
  

  # for ubuntu
  sudo apt update
  sudo apt install portaudio19-dev
  sudo apt install ffmpeg
  

  Note:

  • ffmpeg>=4.4 is needed to work with torchaudio>=2.1.0

  • Mac users may need to add ffmpeg library path to DYLD_LIBRARY_PATH for torchaudio to work:

    export DYLD_LIBRARY_PATH=/opt/homebrew/lib:$DYLD_LIBRARY_PATH
    

  Then install all python requirements

  pip install -r requirements.txt
  

  If you need a faster local speech to text, install whisperX

  pip install git+https://github.com/m-bain/whisperx.git
  

• Step 3. Create an empty sqlite database if you have not done so before

  sqlite3 test.db "VACUUM;"
  

• Step 4. Run db upgrade

  alembic upgrade head
  

  This ensures your database schema is up to date. Please run this after every time you pull the main branch.

• Step 5. Setup .env:

  cp .env.example .env
  

  Update API keys and configs following the instructions in the .env file.

  Note that some features require a working login system. You can get your own OAuth2 login for free with Firebase if needed. To enable, set USE_AUTH to true and fill in the FIREBASE_CONFIG_PATH field. Also fill in Firebase configs in client/next-web/.env.

• Step 6. Run backend server with cli.py or use uvicorn directly

  python cli.py run-uvicorn
  # or
  uvicorn realtime_ai_character.main:app
  

• Step 7. Run frontend client:

  • web client:

    Create an .env file under client/next-web/

    cp client/next-web/.env.example client/next-web/.env
    

    Adjust .env according to the instruction in client/next-web/README.md.

    Start the frontend server:

    python cli.py next-web-dev
    # or
    cd client/next-web && npm run dev
    # or
    cd client/next-web && npm run build && npm run start
    

    After running these commands, a local development server will start, and your default web browser will open a new tab/window pointing to this server (usually http://localhost:3000).

  • (Optional) Terminal client:

    Run the following command in your terminal

    python client/cli.py
    

  • (Optional) mobile client:

    open client/mobile/ios/rac/rac.xcodeproj/project.pbxproj in Xcode and run the app

• Step 8. Select one character to talk to, then start talking. Use GPT4 for better conversation and Wear headphone for best audio(avoid echo)

Note if you want to remotely connect to a RealChar server, SSL set up is required to establish the audio connection.

👨‍🚀 API Keys and Configurations

1. LLMs

1.1 ReByte API Key

To get your ReByte API key, follow these steps:

1. Go to the ReByte website and sign up for an account if you haven't already.
2. Once you're logged in, go to Settings > API Keys.
3. Generate a new API key by clicking on the "Generate" button.

1.2 (Optional) OpenAI API Token

<details><summary>👇click me</summary> This application utilizes the OpenAI API to access its powerful language model capabilities. In order to use the OpenAI API, you will need to obtain an API token.

To get your OpenAI API token, follow these steps:

1. Go to the OpenAI website and sign up for an account if you haven't already.
2. Once you're logged in, navigate to the API keys page.
3. Generate a new API key by clicking on the "Create API Key" button.

(Optional) To use Azure OpenAI API instead, refer to the following section:

1. Set API type in your .env file: OPENAI_API_TYPE=azure

If you want to use the earlier version 2023-03-15-preview:

`OPENA