CyberVerse

<h1 align="center">CyberVerse</h1> <p align="center"><em>CyberVerse is an open-source <strong>real-time digital-human Agent framework</strong>. It uses WebRTC, persona memory, tools, RAG, and optional digital-human video capabilities to help you build AI agents centered on voice interaction.</em></p> <p align="center"> <a href="README.md"><strong>English</strong></a> · <a href="README.zh-CN.md">简体中文</a> · <a href="README.ja.md">日本語</a> · <a href="README.ko.md">한국어</a> </p> <p align="center"> <a href="LICENSE"><img src="https://img.shields.io/badge/License-GPL%20v3-blue.svg" alt="License: GPL v3"/></a> <a href="https://github.com/dsd2077/CyberVerse/pulls"><img src="https://img.shields.io/badge/PRs-welcome-brightgreen.svg" alt="PRs Welcome"/></a> <a href="https://deepwiki.com/dsd2077/CyberVerse"><img src="https://deepwiki.com/badge.svg" alt="Ask DeepWiki" /></a> <a href="https://x.com/dsd2077"><img src="https://img.shields.io/badge/@dsd2077-black?logo=x&logoColor=white" alt="X" /></a> </p> <p align="center"> <a href="docs/assets/logo.png"><img src="docs/assets/logo.png" alt="CyberVerse logo" width="100%"/></a> </p>

Sponsor

<details open> <summary>Click to collapse</summary> <table> <tr> <td width="180"><a href="https://passport.compshare.cn/register?referral_code=IBmJcGPVu1RF78dMihkQCX"><img src="https://www.compshare.cn/logo-compshare.png" alt="Compshare" width="150"></a></td> <td>Thanks to Compshare (优云智算) for sponsoring CyberVerse! Compshare is UCloud's AI cloud platform, offering on-demand GPU rental and model API services. <strong>Its core GPU rental service provides rapidly provisioned GPU instances with usage-based billing for model, algorithm, and application development.</strong> Compshare also provides one-stop access to domestic and international models, with support for Claude Code, Codex, and direct API use. <a href="https://passport.compshare.cn/register?referral_code=IBmJcGPVu1RF78dMihkQCX">Register through this invitation link</a>.</td> </tr> </table> </details>

---

One Photo. A Living Digital Human.

Ever dreamed of having your own J.A.R.V.I.S. — an AI that truly sees you, hears you, and talks back in real time?

Want to see someone you've lost again, hear their voice, watch them smile at you?

Or maybe there's a character you've always wished you could bring to life?

Just one photo. CyberVerse makes them alive.

What is a Digital-Human Agent?

<p align="center"> <a href="docs/assets/digital-human-agent.jpeg"><img src="docs/assets/digital-human-agent.jpeg" alt="CyberVerse digital-human Agent" width="100%"/></a> </p>

Demo

<p align="center"><em>The following characters are demo examples only. They are not bundled with CyberVerse and are not provided for commercial use.</em></p> <p align="center"> <a href="docs/assets/character1.png"><img src="docs/assets/character1.png" alt="CyberVerse character selection gallery" width="100%"/></a> </p> <p align="center"> <a href="docs/assets/character2.png"><img src="docs/assets/character2.png" alt="CyberVerse character gallery examples" width="100%"/></a> </p> <div align="center">

| | | |:---:|:---:| | Alice — watch on YouTube | Lina — watch on YouTube |

| | |:---:| | Xiaolongnü — watch on YouTube |

</div>

Features

Realtime Digital Human Video Interaction

With just one photo, you can create a digital human ready for real-time video conversation. Users can interact as naturally as a video call with a real person, interrupting or speaking over the digital human at any time for a full-duplex realtime experience.

CyberVerse integrates the local FlashHead and LiveAct digital-human models, and supports cloud digital-human offerings such as Baidu Xiling and Xunfei Digital Human, covering a strong set of current open-source and commercial digital-human options.

| Model | Quality | GPU | Count | Resolution | FPS | Real-time? | |-------|---------|-----|-------|------------|-----|------------| | FlashHead 1.3B | Pro | RTX 5090 | 2 | 512×512 | 25+ | ✅ Yes | | FlashHead 1.3B | Pro | RTX 5090 | 1 | 464x464 | 20 | ✅ Yes | | LiveAct 18B | — | RTX PRO 6000 | 2 | 320×480 | 20 | ✅ Yes | | LiveAct 18B | — | RTX PRO 6000 | 1 | 256×417 | 20 | ✅ Yes | | Vidu S1 | Cloud API | No local GPU required | — | 540P | 25 | ✅ Yes | | Baidu Xiling Digital Human | Cloud API | No local GPU required | — | Provider/figure config | Provider response | ✅ Yes | | Xunfei Digital Human | Cloud API | No local GPU required | — | Provider/figure config | Provider response | ✅ Yes |

PersonaAgent + SubAgent Tasks

CyberVerse uses a multi-agent architecture: PersonaAgent stays in the foreground to maintain fluid conversation, respond quickly to interruptions, and handle context switches; long-running work such as search, research, material organization, summarization, and HTML report generation is delegated to background SubAgents asynchronously.

This keeps complex tasks from slowing down voice turns. Users can keep speaking, ask follow-up questions, or adjust direction, and PersonaAgent can return the SubAgent result once it is ready.

Character Memory and RAG

Each character's conversation history is persisted to local disk and automatically loaded when you re-enter a conversation, preserving continuity across sessions. You can also import knowledge bases, documents, and biographical material for a character; the system indexes them for retrieval-augmented generation, making answers better aligned with the character's background and persona.

Plugin-Based Stack

Brain, voice, hearing, tools, memory, and face are all replaceable modules. Runtime behavior stays in config/cyberverse.yaml, while omni, LLM, TTS, ASR, and embedding provider definitions are loaded from the built-in infra/config/*_models/ directories and optional local overrides under config/*_models/. You can configure different vendors' API keys and service endpoints in the web UI at /settings to switch providers and model combinations by scenario. The LiteLLM plugin adds access to 100+ LLM providers (AWS Bedrock, Azure, Vertex AI, Mistral, Cohere, etc.) through a single unified interface.

Quick Start

Cloud Images

If you want to try CyberVerse quickly without setting up the environment dependencies manually, you can launch it from a cloud image:

• Compshare CyberVerse Image
• AutoDL CyberVerse Image

For local deployment, continue with the installation steps below.

Prerequisites

• Node 18+
• Go 1.25 (required: protoc-gen-go, protoc-gen-go-grpc)
• Conda
• Python 3.10+
• FFmpeg
• libopus-dev、libopusfile-dev、libsoxr-dev，pkg-config

For pure voice sessions, no local avatar GPU is required. Runtime cost depends on the realtime voice/omni/LLM/TTS/ASR providers you configure.

To verify, use:

node --version
go version
protoc --version
ffmpeg -version
conda --version

Step 1: Clone

git clone https://github.com/dsd2077/CyberVerse.git
cd CyberVerse

Step 2: Create Python environment

conda create -n cyberverse python=3.10
conda activate cyberverse

Step 3: Configure environment variables

cp -r infra/config config

Edit config/env and fill in the supported API keys:

Alibaba Cloud Qwen-series models:

DASHSCOPE_API_KEY=your_dashscope_api_key

Or Volcengine Doubao-series models:

DOUBAO_ACCESS_TOKEN=your_doubao_access_token
DOUBAO_APP_ID=your_doubao_app_id

Doubao Voice: follow the Volcengine quick start to get App ID / API Key, then fill in DOUBAO_APP_ID / DOUBAO_ACCESS_TOKEN.

After the stack is running, you can change API keys and service endpoints from the web UI at /settings instead of editing config/env only.

Omni, LLM, embedding, TTS, and ASR model definitions are discovered automatically from infra/config/*_models/. Create matching files under config/*_models/ only when you want local overrides.

Step 4: Create local config and enable voice-only mode

Edit config/cyberverse.yaml:

inference:
  avatar:
    enabled: false

With enabled: false, CyberVerse runs as a pure voice agent assistant.

Step 5: Install project dependencies

make setup

This installs the base editable package ([dev,inference]), generates gRPC stubs, and installs frontend dependencies.

Install the voice-agent extras used by the default config:

# all optional groups at once
pip install -e ".[all]"

Step 6: Start services (3 terminals)

Terminal 1 — Python inference server:

conda activate cyberverse
make inference

Terminal 2 — Go API server:

make server

Terminal 3 — Frontend:

make frontend

Step 7: Verify

# Check API health
curl -s http://localhost:8080/api/v1/health

Open http://localhost:5173 in your browser.

Optional: Full Digital-Human Video

If you want to drive realtime Avatar video with FlashHead or LiveAct, follow the steps below.

Additional Requirements

• GPU with CUDA 12.8+
• PyTorch 2.8 (CUDA 12.8)
• FFmpeg with libvpx for video encoding
• Avatar model weights

Install PyTorch (CUDA 12.8):

pip3 install torch==2.8.0 torchvision==0.23.0 torchaudio==2.8.0 --index-url https://download.pytorch.org/whl/cu128

Install vllm if you use LiveAct:

p