VibeVoice ComfyUI Nodes

A comprehensive ComfyUI integration for Microsoft's VibeVoice text-to-speech model, enabling high-quality single and multi-speaker voice synthesis directly within your ComfyUI workflows.

✨ Features

Core Functionality

• 🎤 Single Speaker TTS: Generate natural speech with optional voice cloning
• 👥 Multi-Speaker Conversations: Support for up to 4 distinct speakers
• 🎯 Voice Cloning: Clone voices from audio samples
• 🎨 LoRA Support: Fine-tune voices with custom LoRA adapters (v1.4.0+)
• 🎚️ Voice Speed Control: Adjust speech rate by modifying reference voice speed (v1.5.0+)
• 📝 Text File Loading: Load scripts from text files
• 📚 Automatic Text Chunking: Handles long texts seamlessly with configurable chunk size
• ⏸️ Custom Pause Tags: Insert silences with [pause] and [pause:ms] tags (wrapper feature)
• 🔄 Node Chaining: Connect multiple VibeVoice nodes for complex workflows
• ⏹️ Interruption Support: Cancel operations before or between generations
• 🔧 Flexible Configuration: Control temperature, sampling, and guidance scale

Performance & Optimization

• ⚡ Attention Mechanisms: Choose between auto, eager, sdpa, flash_attention_2 or sage
• 🎛️ Diffusion Steps: Adjustable quality vs speed trade-off (default: 20)
• 💾 Memory Management: Toggle automatic VRAM cleanup after generation
• 🧹 Free Memory Node: Manual memory control for complex workflows
• 🍎 Apple Silicon Support: Native GPU acceleration on M1/M2/M3 Macs via MPS
• 🔢 8-Bit Quantization: Perfect audio quality with high VRAM reduction
• 🔢 4-Bit Quantization: Maximum VRAM savings with minimal quality loss

Compatibility & Installation

• 📦 Self-Contained: Embedded VibeVoice code, no external dependencies
• 🔄 Universal Compatibility: Adaptive support for transformers v4.51.3+
• 🖥️ Cross-Platform: Works on Windows, Linux, and macOS
• 🎮 Multi-Backend: Supports CUDA, CPU, and MPS (Apple Silicon)

🎥 Video Demo

<p align="center"> <a href="https://www.youtube.com/watch?v=fIBMepIBKhI"> <img src="https://img.youtube.com/vi/fIBMepIBKhI/maxresdefault.jpg" alt="VibeVoice ComfyUI Wrapper Demo" /> </a> <br> <strong>Click to watch the demo video</strong> </p>

📦 Installation

Automatic Installation (Recommended)

1. Clone this repository into your ComfyUI custom nodes folder:

cd ComfyUI/custom_nodes
git clone https://github.com/Enemyx-net/VibeVoice-ComfyUI

2. Restart ComfyUI - the nodes will automatically install requirements on first use

📥 Model Installation

Manual Download Required

Starting from version 1.6.0, models and tokenizer must be manually downloaded and placed in the correct folder. The wrapper no longer downloads them automatically.

Download Links

Models

You can download VibeVoice models from HuggingFace:

| Model | Size | Download Link | |------------------------|--------|---------------| | VibeVoice-1.5B | ~5.4GB | microsoft/VibeVoice-1.5B | | VibeVoice-Large | ~18.7GB | aoi-ot/VibeVoice-Large | | VibeVoice-Large-Q8 | ~11.6GB | FabioSarracino/VibeVoice-Large-Q8 | | VibeVoice-Large-Q4 | ~6.6GB | DevParker/VibeVoice7b-low-vram |

Tokenizer (Required)

VibeVoice uses the Qwen2.5-1.5B tokenizer:

• Download from: Qwen2.5-1.5B Tokenizer
• Required files: tokenizer_config.json, vocab.json, merges.txt, tokenizer.json

Installation Steps

1. Create the models folder if it doesn't exist:

   ComfyUI/models/vibevoice/
   

2. Download and organize files in the vibevoice folder:

   ComfyUI/models/vibevoice/
   ├── tokenizer/                 # Place Qwen tokenizer files here
   │   ├── tokenizer_config.json
   │   ├── vocab.json
   │   ├── merges.txt
   │   └── tokenizer.json
   ├── VibeVoice-1.5B/           # Model folder
   │   ├── config.json
   │   ├── model-00001-of-00003.safetensors
   │   ├── model-00002-of-00003.safetensors
   │   └── ... (other model files)
   ├── VibeVoice-Large/
   │   └── ... (model files)
   └── my-custom-vibevoice/      # custom names are supported
       └── ... (model files)
   

3. For models downloaded from HuggingFace using git-lfs or the HF CLI, you can also use the cache structure:

   ComfyUI/models/vibevoice/
   └── models--microsoft--VibeVoice-1.5B/
       └── snapshots/
           └── [hash]/
               └── ... (model files)
   

4. Refresh your browser - the models will appear in the dropdown menu

Notes

• The dropdown will show user-friendly names extracted from folder names
• Both regular folders and HuggingFace cache structures are supported
• Models are rescanned on every browser refresh
• Quantized models are automatically detected from their config files
• The tokenizer is searched in this priority order:
  1. ComfyUI/models/vibevoice/tokenizer/ (recommended)
  2. ComfyUI/models/vibevoice/models--Qwen--Qwen2.5-1.5B/ (if exists from previous installations)
  3. HuggingFace cache (if available)

🔧 Available Nodes

1. VibeVoice Load Text From File

Loads text content from files in ComfyUI's input/output/temp directories.

• Supported formats: .txt
• Output: Text string for TTS nodes

2. VibeVoice Single Speaker

Generates speech from text using a single voice.

• Text Input: Direct text or connection from Load Text node
• Models: Select from available models in dropdown menu
• Voice Cloning: Optional audio input for voice cloning
• Parameters (in order):
  • text: Input text to convert to speech
  • model: Select from dropdown list of available models found in ComfyUI/models/vibevoice/
  • attention_type: auto, eager, sdpa, flash_attention_2 or sage (default: auto)
  • quantize_llm: Dynamically quantize only the LLM component for non-quantized models. Options: "full precision" (default), "4bit", or "8bit". 4-bit provides major VRAM savings with minimal quality loss. 8-bit provides a good balance between quality and memory usage. Requires CUDA GPU. Ignored for pre-quantized models.
  • free_memory_after_generate: Free VRAM after generation (default: True)
  • diffusion_steps: Number of denoising steps (5-100, default: 20)
  • seed: Random seed for reproducibility (default: 42)
  • cfg_scale: Classifier-free guidance (1.0-2.0, default: 1.3)
  • use_sampling: Enable/disable deterministic generation (default: False)
• Optional Parameters:
  • voice_to_clone: Audio input for voice cloning
  • lora: LoRA configuration from VibeVoice LoRA node
  • temperature: Sampling temperature (0.1-2.0, default: 0.95)
  • top_p: Nucleus sampling parameter (0.1-1.0, default: 0.95)
  • max_words_per_chunk: Maximum words per chunk for long texts (100-500, default: 250)
  • voice_speed_factor: Speech rate adjustment (0.8-1.2, default: 1.0, step: 0.01)

3. VibeVoice Multiple Speakers

Generates multi-speaker conversations with distinct voices.

• Speaker Format: Use [N]: notation where N is 1-4
• Voice Assignment: Optional voice samples for each speaker
• Recommended Model: VibeVoice-Large for better multi-speaker quality
• Parameters (in order):
  • text: Input text with speaker labels
  • model: Select from dropdown list of available models found in ComfyUI/models/vibevoice/
  • attention_type: auto, eager, sdpa, flash_attention_2 or sage (default: auto)
  • quantize_llm: Dynamically quantize only the LLM component for non-quantized models. Options: "full precision" (default), "4bit", or "8bit". 4-bit provides major VRAM savings with minimal quality loss. 8-bit provides a good balance between quality and memory usage. Requires CUDA GPU. Ignored for pre-quantized models.
  • free_memory_after_generate: Free VRAM after generation (default: True)
  • diffusion_steps: Number of denoising steps (5-100, default: 20)
  • seed: Random seed for reproducibility (default: 42)
  • cfg_scale: Classifier-free guidance (1.0-2.0, default: 1.3)
  • use_sampling: Enable/disable deterministic generation (default: False)
• Optional Parameters:
  • speaker1_voice to speaker4_voice: Audio inputs for voice cloning
  • lora: LoRA configuration from VibeVoice LoRA node
  • temperature: Sampling temperature (0.1-2.0, default: 0.95)
  • top_p: Nucleus sampling parameter (0.1-1.0, default: 0.95)
  • voice_speed_factor: Speech rate adjustment for all speakers (0.8-1.2, default: 1.0, step: 0.01)

4. VibeVoice Free Memory

Manually frees all loaded VibeVoice models from memory.

• Input: audio - Connect audio output to trigger memory cleanup
• Output: audio - Passes through the input audio unchanged
• Use Case: Insert between nodes to free VRAM/RAM at specific workflow points
• Example: [VibeVoice Node] → [Free Memory] → [Save Audio]

5. VibeVoice LoRA

Configure and load custom LoRA adapters for fine-tuned VibeVoice models.

• LoRA Selection: Dropdown menu with available LoRA adapters
• LoRA Location: Place your LoRA folders in ComfyUI/models/vibevoice/loras/
• Parameters:
  • lora_name: Select from available LoRA adapters or "None" to disable
  • llm_strength: Strength of the language model LoRA (0.0-2.0, default: 1.0)
  • use_llm: Apply language model LoRA component (default: True)
  • use_diffusion_head: Apply diffusion head replacement (default: True)
  • use_acoustic_connector: Apply acoustic connector LoRA (default: True)
  • use_semantic_connector: Apply semantic connector LoRA (default: True)
• Output: lora - LoRA configuration to connect to speaker nodes
• Usage: [VibeVoice LoRA] → [Single/Multiple Speaker Node]

💬 Multi