🪶 MagicQuill: An Intelligent Interactive Image Editing System (CVPR 2025)

<a href="https://magicquill.art/demo/"><img src="https://img.shields.io/static/v1?label=Project&message=magicquill.art&color=blue"></a> <a href="https://arxiv.org/abs/2411.09703"><img src="https://img.shields.io/badge/arXiv-2411.09703-b31b1b.svg"></a> <a href="https://huggingface.co/spaces/AI4Editing/MagicQuill"><img src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Spaces-blue)"></a> <a href="https://creativecommons.org/licenses/by-sa/4.0/"><img src="https://img.shields.io/badge/License-CC%20BY--SA%204.0-lightgrey.svg"></a> <a href="https://magicquill.art/v2/"><img src="https://img.shields.io/static/v1?label=Project&message=MagicQuillV2&color=red"></a>

https://github.com/user-attachments/assets/8ee9663a-fef2-484a-a0b7-8427ab590424

There is an HD video on Youtube.

Zichen Liu^*,1,2, Yue Yu^*,1,2, Hao Ouyang², Qiuyu Wang², Ka Leong Cheng^1,2, Wen Wang^3,2, Zhiheng Liu⁴, Qifeng Chen^†,1, Yujun Shen^†,2<br> ¹HKUST ²Ant Group ³ZJU ⁴HKU ^*equal contribution ^†corresponding author

TLDR: MagicQuill is an intelligent and interactive system achieving precise image editing.

Key Features: 😎 User-friendly interface / 🤖 AI-powered suggestions / 🎨 Precise local editing

• 🪶 MagicQuill: An Intelligent Interactive Image Editing System
  • TODO List
  • Update Log
  • Hardware Requirements
  • Docker Container
  • Setup
  • Tutorial
  • Citation
  • Acknowledgement
  • Note

MagicQuillV2 has been released!!! Check it out ;)

TODO List

• [x] Release the paper and demo page. Visit magicquill.art 🪩
• [x] Release the code and checkpoints.
• [x] Release gradio demo.
• [x] Release ComfyUI MagicQuill custom node.

<img src="docs/comfyui.png" width="100%" >

Update Log

• [2024.11.21] 📢 Update the save button; Fix path bug on Windows; Add .bat and .sh files for convenient environment install on Windows and Linux. Thanks lior007 and JamesIV4.
• [2024.11.25] 📢 New UI Updates: Drag & Drop Images + Download Button: We've enhanced our interface with two exciting features! Now you can easily upload images with drag & drop functionality, and quickly save your work using our new download button. Try it out and let us know what you think!
• [2024.12.06] 📢 New Feature Updates: Auto-save and Resolution Adjustment are now enabled in the parameter settings. Thanks Furkan Gözükara for his brilliant suggestions.
• [2024.12.07] 🎉 Exciting News: ComfyUI MagicQuill Node has been released! Check the repository https://github.com/magic-quill/ComfyUI_MagicQuill for more details.
• [2024.12.16] 🎉 Exciting News: MagicQuill is also available at Modelscope. Thanks for their amazing support and infrastructure.
• [2025.01.02] 🎉 Exciting News: MagicQuill docker container is now available. You can now build & run your own image in a cleaner, isolated environment. Thanks gbudge for his contribution.
• [2025.02.27] 🎉 Exciting News: MagicQuill has been accepted to CVPR 2025! Looking forward to meeting everyone in Tennessee!
• [2025.12.03] 🎉 Exciting News: MagicQuillV2 has been released! Check it out ;), give us a star if you are interested.

To update the latest features, pull the latest code and re-install the gradio_magicquill:

pip uninstall -y gradio_magicquill-0.0.1-py3-none-any.whl
pip install gradio_magicquill-0.0.1-py3-none-any.whl

Hardware Requirements

• GPU is required to run MagicQuill. Through our testing, we have confirmed that the model can run on GPUs with 8GB VRAM (RTX4070 Laptop).

For users with limited GPU resources, please try our Huggingface Demo and Modelscope Demo. Also, consider disabling the DrawNGuess automatic prompt filling by clicking the wand icon above if it takes too long time on your machine.

Setup

Follow the following guide to set up the environment.

1. git clone repo. Please don't forget the --recursive flag. Otherwise, you will find LLaVA submodule missing.

   git clone --recursive https://github.com/magic-quill/MagicQuill.git
   cd MagicQuill
   

2. download and unzip checkpoints

   wget -O models.zip "https://hkustconnect-my.sharepoint.com/:u:/g/personal/zliucz_connect_ust_hk/EWlGF0WfawJIrJ1Hn85_-3gB0MtwImAnYeWXuleVQcukMg?e=Gcjugg&download=1"
   unzip models.zip
   

   If the .zip file is not accessible, download it via browser. All checkpoints are about 25 GB in total. It may take some time to download. Alternatively, check our checkpoints at huggingface.

---

If you are a Windows user, you may try to use windows_setup.bat to conveniently install environments, just enter windows_setup.bat in your Python shell. For Linux user, check linux_setup.sh.

Alternatively, follow the step-by-step installation guide.

3. create environment

   conda create -n MagicQuill python=3.10 -y
   conda activate MagicQuill
   

4. install torch with GPU support

   pip install torch==2.1.2 torchvision==0.16.2 torchaudio==2.1.2 --index-url https://download.pytorch.org/whl/cu118
   

5. install the interface

   pip install gradio_magicquill-0.0.1-py3-none-any.whl
   

6. install llava environment

   (For Linux)
   cp -f pyproject.toml MagicQuill/LLaVA/
   pip install -e MagicQuill/LLaVA/
   

   or

   (For Windows)
   copy /Y pyproject.toml MagicQuill\LLaVA\
   pip install -e MagicQuill\LLaVA\
   

   (For Windows PowerShell, the first line should be Copy-Item -Path pyproject.toml -Destination "MagicQuill\LLaVA" -Force)

7. install the remaining environment

   pip install -r requirements.txt
   

8. run magicquill

   python gradio_run.py
   

   If you are mainland user, you may try export HF_ENDPOINT=https://hf-mirror.com to use huggingface mirror to facilitate the download of some necessary checkpoints to run our system.

Docker Container

You can build a docker container with MagicQuill as follows:

1. git clone repo. Please don't forget the --recursive flag. Otherwise, you will find LLaVA submodule missing.

   git clone --recursive https://github.com/magic-quill/MagicQuill.git
   cd MagicQuill
   

2. download and unzip checkpoints

   wget -O models.zip "https://hkustconnect-my.sharepoint.com/:u:/g/personal/zliucz_connect_ust_hk/EWlGF0WfawJIrJ1Hn85_-3gB0MtwImAnYeWXuleVQcukMg?e=Gcjugg&download=1"
   unzip models.zip
   

   If the .zip file is not accessible, download it via browser. All checkpoints are about 25 GB in total. It may take some time to download. Alternatively, check our checkpoints at huggingface.

   Note: these can be located anywhere on the host computer, but Docker Compose expects them to be in /data/magicquill/models by default. Update docker-compose.yaml if you unzip them to another location.

3. build the image

   docker compose build
   

4. run the image

   docker compose up -d
   

Tutorial

Please read before you try!

<!DOCTYPE html> <html> <body> <div class="tutorial"><div><h3 align="center" class="heading">I. Three type of magic quills</h3><div align="center"><img fill="white" src="docs/icons/brush_edge_add.svg" alt="SVG image" class="icon" width="100"></div><div align="center"><br>Use the <b>add brush</b> to add details and elements guided by prompts - express your ideas with your own lively strokes!<br></div><div class="ant-row css-1kuana8"><div class="ant-col ant-col-xs-24 ant-col-md-12 css-1kuana8"><div align="center"><br><img width="300" src="docs/gifs/deer.gif" alt="gif description" class="gif"></div><div align="center"><small>"With just a few strokes, a vivid little deer comes to life"</small><br></div></div><div class="ant-col ant-col-xs-24 ant-col-md-12 css-1kuana8"><div align="center"><br><img width="300" src="docs/gifs/necklace.gif" alt="gif description" class="gif"></div><div align="center"><small> "Adorn the beautiful lady with a necklace"</small><br></div></div></div><div align="center"><br><img src="docs/icons/brush_edge_remove.svg" alt="SVG image" class="icon" width="100"></div><div align="center"><br>The <b>subtract brush</b> can remove excess details or redraw areas based on prompts. If there's anything you're not satisfied with, just subtract it away!<br></div><div class="ant-row css-1kuana8"><div class="ant-col ant-col-xs-24 ant-col-md-12 css-1kuana8"><div align="center"><br><img width="300" src="docs/gifs/dolphin.gif" alt="gif description" class="gif"></div><div align="center"><small> "A dolphin with two tail fins? Let's give it a quick 'treatment'!"</small><br></div></div><div class="ant-col ant-col-xs-24 ant-col-md-12 css-1kuana8"><div align="center"><br><img width="300" src="docs/gifs/skeleton cowboy.gif" alt="gif descrip