Gic

<div align="center">

Gaussian-Informed Continuum for Physical Property Identification and Simulation (NeurIPS 2024 Oral)

Junhao Cai^1*, Yuji Yang^2*, Weihao Yuan^3†, Yisheng He³, Zilong Dong³, Liefeng Bo³, Hui Cheng², Qifeng Chen¹

<p>¹The Hong Kong University of Science and Technology, &nbsp;&nbsp;²Sun Yat-sen University, &nbsp;&nbsp;³Alibaba Group &nbsp;&nbsp; <br>^*Equal contribution, order determined by coin toss. &nbsp;&nbsp; <br>^†Corresponding author &nbsp;&nbsp; <br>Accepted at Conference on Neural Information Processing Systems, 2024 (Oral) &nbsp;&nbsp;

Project Page | Paper

</div>

Environment

The code has been tested on Ubuntun 22.04 with CUDA Tookit 12.1. Please ensure that you have correctly installed the CUDA Toolkit 12.1 before running the following commands to install the remaining dependencies.

git clone https://github.com/Jukgei/gic
cd gic

conda env create --file environment.yml
conda activate gic

Dataset

We used the same segmentation model as PAC-NeRF for segmentation. We validated our algorithm using the PAC-NeRF dataset and the Spring-Gaus dataset.

The structure of the dataset and model files in the root directory is as follows:

├── data
│   | checkpoint
│     ├── pytorch_resnet101.pth
│   | pacnerf
│     ├── bird
│     ├── cat
│     ├── ...
│   | sgs
│     ├── apple
│     ├── C
│     ├── cream
│     ├── ...

Segmentation

If you're running it for the first time, you need to perform foreground-background segmentation on the dataset first.

python prepare_pacnerf_data.py --data_folder=data/pacnerf/torus

Training

python train_dynamic.py -c config/pacnerf/torus.json -s data/pacnerf/torus -m output/pacnerf/torus --reg_scale --reg_alpha

The results of the parameter estimation and the related settings will be saved in the {config_id}-pred.json file located in the output/pacnerf/torus directory.

Application

After estimating the parameters, we can modify different materials or physical parameters (such as gravity, Young's modulus, etc.) to generate new videos. For example, For example, we change the Young's modulus of the torus.

python new_trajectory.py -c config/predict/elastic.json -s data/pacnerf/torus -m output/pacnerf/torus -vid 3 -cid 0 --reg_alpha

Command-Line Arguments Explanation

• vid: camera id.

• cid: estimated result file id.

The new video output.mp4 will be generated in the output/pacnerf/torus directory.

Acknowledgments

We sincerely thank the authors of Deformable Gaussians, PAC-NeRF, Spring-Gaus, whose codes and datasets were used in our work.

BibTex

@article{cai2024gaussian,
  title={Gaussian-Informed Continuum for Physical Property Identification and Simulation},
  author={Cai, Junhao and Yang, Yuji and Yuan, Weihao and He, Yisheng and Dong, Zilong and Bo, Liefeng and Cheng, Hui and Chen, Qifeng},
  journal={Advances in Neural Information Processing Systems},
  year={2024}
}