SVGDreamer: Text Guided SVG Generation with Diffusion Model

<div align="center" style="line-height: 1;">

</div>

This repository contains the official implementation of our CVPR 2024 paper, "SVGDreamer: Text-Guided SVG Generation with Diffusion Model." The method leverages a diffusion-based approach to produce high-quality SVGs guided by text prompts.

:new: Latest Update

• [11/2024] 🔥 We released the SVGDreamer++, offering stronger visual representation and improved editing capabilities.
• [03/2024] 🔥 We released the code for SVGDreamer.
• [02/2024] 🎉 SVGDreamer accepted by CVPR2024. 🎉
• [12/2023] 🔥 We released the SVGDreamer Paper. SVGDreamer is a novel text-guided vector graphics synthesis method. This method considers both the editing of vector graphics and the quality of the synthesis.

📌 Installation Guide

🛠️ Step 1: Set Up the Environment

To quickly get started with SVGDreamer, follow the steps below.
These instructions will help you run quick inference locally.

🚀 Option 1: Standard Installation

Run the following command in the top-level directory:

chmod +x script/install.sh
bash script/install.sh

🐳 Option 2: Using Docker

chmod +x script/run_svgdreamer_docker.sh
sudo bash script/run_svgdreamer_docker.sh

🛠️ Step 2: Download Pretrained Stable Diffusion Model

SVGDreamer requires a pretrained Stable Diffusion (SD) model. You can download it automatically or manually.

🔄 Option 1: Auto-Download (Recommended)

Set diffuser.download=True in /conf/config.yaml before running SVGDreamer. Alternatively, append diffuser.download=True to the execution script.

⬇️ Option 2: Manual Download

If you prefer manual setup, download the model from Hugging Face:

🔗 Model Link: Stable Diffusion 2.1 Base

The model will be stored at:

📁 Default Path: ~/.cache/huggingface/hub/models--stabilityai--stable-diffusion-2-1-base

🔥 Quickstart: synthesize 6 SVGs at once

SIVE + VPSD

Prompt: an image of Batman. full body action pose, complete detailed body, white background, high quality, 4K, ultra realistic <br/> Preview:

| Particle 1 | Particle 2 | Particle 3 | Particle 4 | Particle 5 | Particle 6 | |:---------------------------------------------:|:---------------------------------------------:|:---------------------------------------------:|:--------------------------------------------:|:---------------------------------------------:|:---------------------------------------------:| | init p1 | init p2 | init p3 | init p4 | init p5 | init p6 | | <img src="./assets/case-batman/init_p0.svg"> | <img src="./assets/case-batman/init_p1.svg"> | <img src="./assets/case-batman/init_p2.svg"> | <img src="./assets/case-batman/init_p3.svg"> | <img src="./assets/case-batman/init_p4.svg"> | <img src="./assets/case-batman/init_p5.svg"> | | final p1 | final p2 | final p3 | final p4 | final p5 | final p6 | | <img src="./assets/case-batman/final_p0.svg"> | <img src="./assets/case-batman/final_p1.svg"> | <img src="./assets/case-batman/final_p2.svg"> | <img src="assets/case-batman/final_p3.svg"> | <img src="./assets/case-batman/final_p4.svg"> | <img src="./assets/case-batman/final_p5.svg"> |

Script:

python svgdreamer.py x=iconography skip_sive=False "prompt='an image of Batman. full body action pose, complete detailed body. white background. empty background, high quality, 4K, ultra realistic'" token_ind=4 x.vpsd.t_schedule='randint' result_path='./logs/batman' multirun=True

🔹Parameter:

• x=iconography(str): style configs
• skip_sive(bool): enable the SIVE stage
• token_ind(int): the index of text prompt, from 1
• result_path(str): the path to save the result
• multirun(bool): run the script multiple times with different random seeds
• mv(bool): save the intermediate results of the run and record the video (This increases the run time)

More parameters in ./conf/x/style.yaml, you can modify these parameters from the command line. For example, append x.vpsd.n_particle=4 to the end of the script.

SIVE

Prompt: an astronaut walking across a desert, planet mars in the background, floating beside planets, space art <br/> Preview:

| attn-map | bg init | fg init | bg final | fg final | final | |:----------------------------------------------:|:-------------------------------------------------:|:-------------------------------------------------:|:--------------------------------------------------:|:--------------------------------------------------:|:------------------------------------------------:| | <img src="./assets/SIVE-astronaut-1/attn.png"> | <img src="./assets/SIVE-astronaut-1/init_bg.svg"> | <img src="./assets/SIVE-astronaut-1/init_fg.svg"> | <img src="./assets/SIVE-astronaut-1/bg_final.svg"> | <img src="./assets/SIVE-astronaut-1/fg_final.svg"> | <img src="./assets/SIVE-astronaut-1/result.svg"> |

Script:

python svgdreamer.py x=iconography-s1 skip_sive=False "prompt='an astronaut walking across a desert, planet mars in the background, floating beside planets, space art'" token_ind=5 result_path='./logs/astronaut_sive' seed=116740

VPSD

✍️ Iconography style

Prompt: Sydney opera house. oil painting. by Van Gogh <br/> Preview:

| Particle 1 | Particle 2 | Particle 3 | Particle 4 | Particle 5 | Particle 6 | |:------------------------------------------------------:|:------------------------------------------------------:|:------------------------------------------------------:|:------------------------------------------------------:|:------------------------------------------------------:|:------------------------------------------------------:| | init p1 | init p2 | init p3 | init p4 | init p5 | init p6 | | <img src="./assets/Icon-SydneyOperaHouse/init_p0.svg"> | <img src="./assets/Icon-SydneyOperaHouse/init_p1.svg"> | <img src="./assets/Icon-SydneyOperaHouse/init_p2.svg"> | <img src="./assets/Icon-SydneyOperaHouse/init_p3.svg"> | <img src="./assets/Icon-SydneyOperaHouse/init_p4.svg"> | <img src="./assets/Icon-SydneyOperaHouse/init_p5.svg"> | | final p1 | final p2 | final p3 | final p4 | final p5 | final p6 | | <img src="./assets/Icon-SydneyOperaHouse/p_0.svg"> | <img src="./assets/Icon-SydneyOperaHouse/p_1.svg"> | <img src="./assets/Icon-SydneyOperaHouse/p_2.svg"> | <img src="assets/Icon-SydneyOperaHouse/p_3.svg"> | <img src="./assets/Icon-SydneyOperaHouse/p_4.svg"> | <img src="./assets/Icon-SydneyOperaHouse/p_5.svg"> |

Script:

python svgdreamer.py x=iconography "prompt='Sydney opera house. oil painting. by Van Gogh'" result_path='./logs/SydneyOperaHouse-OilPainting' state.mprec='fp16'

✍️ Painting style

Prompt: Abstract Vincent van Gogh Oil Painting Elephant, featuring earthy tones of green and brown <br/> Preview:

| Particle 1 | Particle 2 | Particle 3 | Particle 4 | Particle 5 | Particle 6 | |:---