PeRF

<div align="center"> <h1>PERF: Panoramic Neural Radiance Field from a Single Panorama</h1> <div> TPAMI 2024 </div> <div> <a href='https://wanggcong.github.io/' target='_blank'>Guangcong Wang*¹</a>&emsp; <a href='https://quartz-khaan-c6f.notion.site/Peng-Wang-0ab0a2521ecf40f5836581770c14219c' target='_blank'>Peng Wang*²</a>&emsp; <a href='https://frozenburning.github.io/' target='_blank'>Zhaoxi Chen¹</a>&emsp; <a href='https://www.cs.hku.hk/people/academic-staff/wenping' target='_blank'>Wenping Wang³</a>&emsp; <a href='https://www.mmlab-ntu.com/person/ccloy/' target='_blank'>Chen Change Loy¹</a>&emsp; <a href='https://liuziwei7.github.io/' target='_blank'>Ziwei Liu¹</a> </div> <div> S-Lab, Nanyang Technological University¹, The University of Hong Kong², Texas A&M University³ </div> <div> * denotes equal contribution </div>

Project | YouTube | arXiv

<div> </div>

<!----> <tr> <img src="img/input_output_one_example_clip.gif" width="90%"/> </tr> </div>

Usage

Setup

Step 1: Clone this repository

git clone https://github.com/perf-project/PeRF.git
cd PeRF
pip install -r requirements.txt

Step 2: Install tiny-cuda-nn

pip install ninja
pip install git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch

Step 3: Download checkpoints as shown here.

Train

Here is a command to train a PeRF of an example data:

python core_exp_runner.py --config-name nerf dataset.image_path=$(pwd)/example_data/kitchen/image.png device.base_exp_dir=$(pwd)/exp

Render a video

After training is done, you can render a traverse video by running the following command:

python core_exp_runner.py --config-name nerf dataset.image_path=$(pwd)/example_data/kitchen/image.png device.base_exp_dir=$(pwd)/exp mode=render_dense is_continue=true

Citation

Cite as below if you find it helpful to your research.

@article{perf2023,
    title={PERF: Panoramic Neural Radiance Field from a Single Panorama},
    author={Guangcong Wang and Peng Wang and Zhaoxi Chen and Wenping Wang and Chen Change Loy and Ziwei Liu},
    journal={IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)},
    year={2024}}