📜 Fillerbuster: Multi-View Scene Completion for Casual Captures

Fillerbuster is a unified multi-view diffusion model for solving a variety of scene completion tasks. We trained Fillerbuster from scratch. Here we provide training code and inference code for completing scenes.

Project Page @ https://ethanweber.me/fillerbuster

We provide training code and inference code, but we do not provide training data. However, this codebase should be easy to adapt to multi-view datasets!

Set Up Environment

1. Create and activate the fillerbuster environment:

   conda create -n fillerbuster python=3.10 -y
   conda activate fillerbuster
   

2. Install dependencies and the repo

   pip install torch torchvision --index-url https://download.pytorch.org/whl/124
   pip install -e .
   

Download Weights and Inference Data

1. Download Fillerbuster weights from this folder and place inside a checkpoints folder.

2. Download CLIP weights into the checkpoints folder.

   git lfs install
   cd checkpoints
   git clone https://huggingface.co/openai/clip-vit-large-patch14
   

3. Download casual capture data from other works: namely, LERF, Nerfbusters, NeRFiller, and Nerfstudio datasets from this convenient all-on-one folder. Download and extract the data.zip file to create a data folder as specified in their README.txt.

4. Download our videos and place them in data/videos.

Demo

We provide demo.ipynb as a minimal example to run inference with our model.

Experiments

If you want more details on experiments, see EXPERIMENTS.md. Here we provide a minimal set of commands to get started!

1. "Completing Casually Captured Scenes" (Nerfbusters data)

   ns-train fillerbuster --data data/nerfbusters-dataset/picnic --output-dir outputs/nerfstudio-outputs nerfstudio-data --eval-mode filename
   

2. "Uncalibrated Scene Completion" (video data)

   python fillerbuster/scripts/run_uncalibrated_scene_completion.py --data data/videos/couch.mov --output-dir outputs/uncalibrated-outputs
   

3. "Completing Masked 3D Regions" (NeRFiller data)

   ns-train fillerbuster --data data/nerfiller-dataset/billiards --output-dir outputs/nerfstudio-outputs --pipeline.inpainter nerfiller --pipeline.dilate-iters 5 --pipeline.context-size 32 --pipeline.densify-num 0 --pipeline.anchor-rotation-num 0 --pipeline.anchor-vertical-num 0
   

Training

This section is not yet operational. Check back soon!

We are releasing our training code. See TRAIN.md for more details. We are not providing the training data.

Citing

If you find this code or data useful for your research, please consider citing the following paper:

@misc{weber2025fillerbuster,
    title = {Fillerbuster: Multi-View Scene Completion for Casual Captures},
    author = {Ethan Weber and Norman M\"uller and Yash Kant and Vasu Agrawal and
        Michael Zollh\"ofer and Angjoo Kanazawa and Christian Richardt},
    note = {arXiv},
    year = {2025},
}

License Notice

Fillerbuster is primarily released under the CC BY-NC license. Please note that some components of this project are governed by alternative license terms:

• diffusers – Apache License 2.0
• nerfstudio – Apache License 2.0
• torchmetrics – Apache License 2.0
• transformers – Apache License 2.0
• dataset_transforms.py - BSD-style License
• discriminator.py - BSD-style License

For additional details, please refer to the respective repositories.