PlanarSplatting

<div align="center"> <h1>PlanarSplatting: Accurate Planar Surface Reconstruction in 3 Minutes</h1>

<a href="https://arxiv.org/abs/2412.03451"><img src="https://img.shields.io/badge/arXiv-2412.03451-b31b1b" alt="arXiv"></a> <a href="https://icetttb.github.io/PlanarSplatting/"><img src="https://img.shields.io/badge/Project_Page-green" alt="Project Page"></a> <a href=""><img src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Demo-blue"></a>

Bin Tan¹, Rui Yu², Yujun Shen¹, Nan Xue¹

¹Ant Group ²University of Louisville

</div>

Official implementation of PlanarSplatting (CVPR 2025 Hightlight)

• An ultrafast method for structured indoor surface reconstruction.
• We support pose-free multi-view inputs based on VGG-T.

<div align="center"> <img src="./assets/pipeline.png" alt="teaser" width="800" /> </div>

📖 Overview

We present PlanarSplatting, an ultra-fast and accurate surface reconstruction approach for multi-view indoor images. We take the 3D planes as the main objective due to their compactness and structural expressiveness in indoor scenes, and develop an explicit optimization framework that learns to fit the expected surface of indoor scenes by splatting the 3D planes into 2.5D depth and normal maps.

⚙️ Installation

1. Clone PlanarSplatting

git clone https://github.com/ant-research/PlanarSplatting.git --recursive 

2. Create the enviroment

conda create -n planarSplatting python=3.10
conda activate planarSplatting

pip install torch==2.3.1 torchvision==0.18.1 torchaudio==2.3.1 --index-url https://download.pytorch.org/whl/cu121
pip install -r requirements.txt 
pip install "git+https://github.com/facebookresearch/pytorch3d.git@stable"

pip install submodules/diff-rect-rasterization
pip install submodules/quaternion-utils

# for running on self-captured images
pip install -e submodules/vggt/

🎯 Quick Start

You can run PlanarSpaltting with an interactive demo for your own data as follows:

python run_gradio_demo.py

Here, we use state-of-the-art VGGT for relative pose and depth estimation and Metric3D v2 for normal estimation.

<div align="center"> <img src="./assets/demo.png" alt="demo" width="800" /> </div>

Alternatively, you can also run PlanarSpaltting without a GUI as follows:

python run_demo.py --data_path path/to/images

🧪 Run on COLMAP data

python run_demo_colmap.py -d path/to/colmap/data

🧪 Run on the ScanNetV2 scenes

1. Download ScanNetv2

Please follow instructions reported in SimpleRecon to download and extract ScanNetV2 scenes. The extacted data format should be like:

data/ScanNetV2
    scans/
        scene0000_00/
            scene0000_00_vh_clean_2.ply (gt mesh)
            sensor_data/
                frame-000000.pose.txt
                frame-000000.color.jpg 
            scene0000.txt (scan metadata and image sizes)
            intrinsic/
                intrinsic_depth.txt
                intrinsic_color.txt
        scene0000_01/
        ....

2. Process scenes for PlanarSplatting

cd planarsplat/data_process/scannetv2
python scannetv2_sample_rgb_intrinsic_pose.py \
    --data_path ../../data/ScanNetV2/ \
    --out_path ../../data/ScanNetV2/processed_scans/ \
    --scene_id scene0488_00

3. Run on one scene

We follow the evaluation methodology from AirPlanes on the ScanNetV2 dataset. For more details, please refer to the AirPlanes repository.

cd planarsplat
python run/runner.py  \
    --base_conf confs/base_conf_planarSplatCuda.conf \
    --conf confs/scannetv2_train.conf \
    --gpu 0 \
    --scan_id scene0488_00;

<details> <summary><span style="font-weight: bold;">Command Line Arguments for runner.py</span></summary>

--base_conf

Path to the base config file.

--conf

Path to the scene specific config file.

--gpu

Index of used GPU.

--scan_id

Name of the scan.

--is_continue

Flag to resume from the latest optimization.

</details> <br>

🧪 Run on the ScanNet++ scenes

1. Download ScanNet++

Please follow instructions reported in ScanNet++ to download and extract ScanNet++ scenes. The extacted data format should be like:

data/ScanNet++
    45b0dac5e3/
        iphone/
        scans/
    16c9bd2e1e/
    ....

2. Process scenes for PlanarSplatting

cd planarsplat/data_process/scannetpp

python scannetpp_prepare_metadata.py \
    --data_path ../../../data/ScanNet++ \
    --out_root_path ../../../data/ScanNetPP/scans \
    --scene_id 45b0dac5e3

python scannetpp_sample_rgb_intrinsic_pose.py \
    --data_path ../../../data/ScanNetPP/ \
    --out_path ../../../data/ScanNetPP/processed_scans/ \
    --scene_id 45b0dac5e3

3. Run on one scene

cd planarsplat
python run/runner.py  \
    --base_conf confs/base_conf_planarSplatCuda.conf \
    --conf confs/scannetpp_train.conf \
    --gpu 0 \
    --scan_id 45b0dac5e3

📜 Citation

If you find our work useful in your research please consider citing our paper:

@inproceedings{PlanarSplatting2024,
    title   = {PlanarSplatting: Accurate Planar Surface Reconstruction in 3 Minutes},
    author  = {Tan, Bin and Yu, Rui and Shen, Yujun and Xue, Nan},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
    year    = {2025}
}

🙏 Acknowledgement

PlanarSplatting is built on the top of several outstanding open-source projects. We are extremely grateful for the contributions of these projects and their communities, whose hard work has greatly propelled the development of the field and enabled our work to be realized.

• 3DGS
• 2DGS
• AirPlanes
• VGGT
• Metric3D v2
• Omnidata
• MoGe