<h1 align="center">OmniPaint: Mastering Object-Oriented Editing via Disentangled Insertion-Removal Inpainting</h1> <h3 align="center">ICCV 2025</h3>  <p align="center"> <a href="https://arxiv.org/pdf/2503.08677"><img src="https://img.shields.io/badge/Paper-arXiv-b31b1b.svg"></a>   <a href="https://www.yongshengyu.com/OmniPaint-Page/"><img src="https://img.shields.io/badge/Website-Project-6535a0"></a>   <a href="https://857109156846ad4e8f.gradio.live/"><img src="https://img.shields.io/badge/Gradio-Demo-FFD21E"></a>   <a href="https://huggingface.co/yeates/OmniPaint"><img src="https://img.shields.io/badge/HuggingFace-Model-FFD21E?logo=huggingface&logoColor=FFD21E"></a>    <a href="https://huggingface.co/datasets/yeates/omnipaint-bench"><img src="https://img.shields.io/badge/HuggingFace-Dataset-FFD21E?logo=huggingface&logoColor=FFD21E"></a> </p> <p align="center"> <img src="assets/removal_demo.gif" style="width:85%;" /> </p> <p align="center"> <img src="assets/insertion_demo.gif" style="width:85%;" /> </p>

Features

🧹 Object Removal - Remove foreground objects and their effects using only object masks
🖼️ Object Insertion - Seamless generative insertion of objects into existing scenes
📊 Novel CFD Metric - Reference-free evaluation of object removal quality

Setup

Install dependencies and download OmniPaint weights:

bash scripts/setup.sh

Run this from the repository root.

Code Usage

Object Removal

Single image:

python scripts/omnipaint_remove.py \
  --input ./demo_assets/removal_samples/images/5.jpg \
  --mask ./demo_assets/removal_samples/masks/5.png \
  --output_dir ./outputs \
  --seed 42 \
  --steps 28 \
  --device cuda:0

Directory:

python scripts/omnipaint_remove.py \
  --input ./demo_assets/removal_samples/images \
  --mask ./demo_assets/removal_samples/masks \
  --output_dir ./outputs \
  --seed 42 \
  --steps 28 \
  --device cuda:0

Object Insertion

Single image:

python scripts/omnipaint_insert.py \
  --background ./demo_assets/insertion_samples/backgrounds/background-2.png \
  --mask ./demo_assets/insertion_samples/masks/mask-2.png \
  --subject ./demo_assets/insertion_samples/subjects/subject-2.png \
  --output_dir ./outputs \
  --seed 42 \
  --steps 28 \
  --device cuda:0 \
  --carvekit_device cuda:0

Directory:

python scripts/omnipaint_insert.py \
  --background ./demo_assets/insertion_samples/backgrounds \
  --mask ./demo_assets/insertion_samples/masks \
  --subject ./demo_assets/insertion_samples/subjects \
  --output_dir ./outputs \
  --seed 42 \
  --steps 28 \
  --device cuda:0 \
  --carvekit_device cuda:0

Demo App

The demo app supports both manual mask drawing and automatic mask generation using segmentation model.

Gradio environment setup:
```
bash scripts/app_setup.sh
```
Run this if you plan to launch app.py. The installation may take around 15 minutes due to SAM2 setup and weight downloads.
Run the app:
```
python app.py
```

Notes

Directory mode expects the following structure by default:
- Removal: demo_assets/removal_samples/images/*.{jpg,png,...} with matching masks in demo_assets/removal_samples/masks/*.{jpg,png,...} (same basenames).
- Insertion: demo_assets/insertion_samples/backgrounds/background-XX.png, masks in demo_assets/insertion_samples/masks/mask-XX.png, subjects in demo_assets/insertion_samples/subjects/subject-XX.png. Direct basename alignment also works.
Mask quality strongly affects insertion performance. Prefer a single connected-component mask; avoid multiple disconnected masks.

Evaluation - CFD Score

See cfd_score for setup and usage.

Citation

If you find OmniPaint useful, please cite our paper:

@inproceedings{yu2025omnipaint,
  title={Omnipaint: Mastering object-oriented editing via disentangled insertion-removal inpainting},
  author={Yu, Yongsheng and Zeng, Ziyun and Zheng, Haitian and Luo, Jiebo},
  booktitle={ICCV},
  pages={17324--17334},
  year={2025}
}

OmniPaint

Install / Use

README