<img alt="image" src='media/lady-gaga.png' height="30px"> Gaga: Group Any Gaussians via 3D-aware Memory Bank

Weijie Lyu, Xueting Li, Abhijit Kundu, Yi-Hsuan Tsai, Ming-Hsuan Yang<br> University of California, Merced - NVIDIA Reaserch - Google DeepMind - Atmanity Inc.

<!-- [](https://hits.seeyoufarm.com) --> <div align='center'> <img alt="image" src='media/teaser.png'> </div>

Gaga groups any Gaussians in an open-world 3D scene and renders multi-view consistent class-agnostic segmentation.<br>

Usage

Please refer to USAGE.md for installation and usage.

Results

🗺️ Open-world 3D Segmentation

MipNeRF 360

https://github.com/weijielyu/Gaga/assets/47323245/62a7ff01-30da-4c5e-ab79-c60c091935c1

Replica

https://github.com/weijielyu/Gaga/assets/47323245/d0d5eece-c838-4be9-a3b9-71d051e97270

ScanNet

https://github.com/weijielyu/Gaga/assets/47323245/f1099e6b-40e1-46af-9c14-f480765065bf

🖌️ Scene Editing

✨ Change the color of cushion on <img src="media/footstool.png" width="50"> to maroon 🟥<br> ✨ Remove <img src="media/stuffed.png" width="50">

https://github.com/weijielyu/Gaga/assets/47323245/803f049f-8930-445c-bc1a-b8bb12df0fbf

Citation

If you find our work useful for your project, please consider citing our paper.

@misc{lyu2024gaga,
      title={Gaga: Group Any Gaussians via 3D-aware Memory Bank}, 
      author={Weijie Lyu and Xueting Li and Abhijit Kundu and Yi-Hsuan Tsai and Ming-Hsuan Yang},
      year={2024},
      eprint={2404.07977},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}