DreamScene

Haoran Li, Haolin Shi, Wenli Zhang, Wenjun Wu, Yong Liao, Lin Wang, Lik-hang Lee, Pengyuan Zhou,Yuli Tian, Kun Lan,Pan Hui,Yuyang Wang,Yonghui Wang,

This repository contains the official implementation for DreamScene: 3D Gaussian-based Text-to-3D Scene Generation via Formation Pattern Sampling.
We extend it in DreamScene: 3D Gaussian-based End-to-end Text-to-3D Scene Generation, with a fully automatic pipeline.

ECCV:

TPAMI:

Note: We compress these motion pictures for faster previewing.

<table class="center"> <tr style="line-height: 0"> <td width=30% style="border: none; text-align: center">A DSLR photo of a modern living room. </td> <td width=30% style="border: none; text-align: center">An autumn park.</td> <td width=30% style="border: none; text-align: center">There are two astronauts on the moon, Sci-fi style</td> </tr> <tr style="line-height: 0"> <td width=30% style="border: none"><img src="assets/livingroom.gif"></td> <td width=30% style="border: none"><img src="assets/autumn_park.gif"></td> <td width=30% style="border: none"><img src="assets/space.gif"></td> </tr> </table>

News

• 2024-07-01: Our paper is accepted by ECCV2024 and to be published!
• 2025-07-29: We released the webpage for our ECCV extension!
• 2025-09-28: Our extended paper has been accepted by TPAMI and will be published soon!

TODO

• [x] Release the code of Formation Pattern Sampling (FPS) for single object.
• [x] Release the code of entire DreamScene for generating dream scenes and our demo video.
• [ ] More samples, and tools for generating layout interactively.
• [ ] Release the code of the improved extension version with full pipeline.

Getting Start!

Requirments

git clone https://github.com/DreamScene-Project/DreamScene.git
cd DreamScene

conda create -n dreamscene python=3.10
conda activate dreamscene

pip install -r requirements.txt -f https://download.pytorch.org/whl/cu118/torch_stable.html

git clone --recursive https://github.com/DreamScene-Project/comp-diff-gaussian-rasterization.git
git clone https://github.com/YixunLiang/simple-knn.git

pip install comp-diff-gaussian-rasterization/
pip install simple-knn/

# Follow https://github.com/facebookresearch/pytorch3d/blob/main/INSTALL.md
pip install "git+https://github.com/facebookresearch/pytorch3d.git@stable"

# Install point-e
git clone https://github.com/crockwell/Cap3D.git
cd Cap3D/text-to-3D/point-e/
pip install -e .

mkdir point_e_model_cache
# Optional: Initialize with better point-e
wget https://huggingface.co/datasets/tiange/Cap3D/resolve/main/misc/our_finetuned_models/pointE_finetuned_with_825kdata.pth
mv pointE_finetuned_with_825kdata.pth point_e_model_cache/
# Modify the parameter init_guided in the configuration file to pointe_825k

# or

wget https://huggingface.co/datasets/tiange/Cap3D/resolve/main/misc/our_finetuned_models/pointE_finetuned_with_330kdata.pth
mv pointE_finetuned_with_330kdata.pth point_e_model_cache/
# Modify the parameter init_guided in the configuration file to pointe_330k

Generate Single Object

python main.py --object --config configs/objects/sample.yaml

Generate Entire Scenes

If your device has more than 40G VRAM, you can run it with a single card. Otherwise, it is recommended to use dual cards.

CUDA_VISIBLE_DEVICES=0,1 python main.py --config configs/scenes/sample_indoor.yaml

CUDA_VISIBLE_DEVICES=2,3 python main.py --config configs/scenes/sample_outdoor.yaml

Acknowledgement

This work is built on many amazing research works and open-source projects, thanks a lot to all the authors for sharing!

• gaussian-splatting and diff-gaussian-rasterization
• luciddreamer
• dreamgaussian
• threestudio

Citation

If you find it useful in your research, please consider citing our papers DreamScene and its extension paper as follows:

@inproceedings{li2024dreamscene,
  title={Dreamscene: 3d gaussian-based text-to-3d scene generation via formation pattern sampling},
  author={Li, Haoran and Shi, Haolin and Zhang, Wenli and Wu, Wenjun and Liao, Yong and Wang, Lin and Lee, Lik-hang and Zhou, Peng Yuan},
  booktitle={European Conference on Computer Vision},
  pages={214--230},
  year={2024},
  organization={Springer}
}

@article{li2025dreamscene,
  title={DreamScene: 3D Gaussian-based End-to-end Text-to-3D Scene Generation},
  author={Li, Haoran and Tian, Yuli and Lan, Kun and Liao, Yong and Wang, Lin and Hui, Pan and Zhou, Peng Yuan},
  journal={arXiv preprint arXiv:2507.13985},
  year={2025}
}