Genesis

<div align="center"> <h3>[🎉NeurIPS 2025!]Genesis: Multimodal Driving Scene Generation with Spatio-Temporal and Cross-Modal Consistency</h3>

Xiangyu Guo^1*, Zhanqian Wu^2*, Kaixin Xiong^2*, Ziyang Xu¹, Lijun Zhou², Gangwei Xu¹, Shaoqing Xu², Haiyang Sun^2†, Bing Wang², Guang Chen², Hangjun Ye², Wenyu Liu¹, Xinggang Wang^1,✉

¹ Huazhong University of Science and Technology ² Xiaomi EV

(*) Equal contribution. (†) Project leader. (✉)Corresponding Author.

<a href="https://arxiv.org/abs/2506.07497"><img src='https://img.shields.io/badge/arXiv-Genesis-red' alt='Paper PDF'></a> <a href="https://xiaomi-research.github.io/genesis/"><img src='https://img.shields.io/badge/Project_Page-Genesis-green' alt='Project Page'></a>

</div> <!-- ## Introduction -->

Abstract

We present Genesis, a unified framework for joint generation of multi-view driving videos and LiDAR sequences with spatio-temporal and cross-modal consistency. Genesis employs a two-stage architecture that integrates a DiT-based video diffusion model with 3D-VAE encoding, and a BEV-aware LiDAR generator with NeRF-based rendering and adaptive sampling. Both modalities are directly coupled through a shared latent space, enabling coherent evolution across visual and geometric domains. To guide the generation with structured semantics, we introduce DataCrafter, a captioning module built on vision-language models that provides scene-level and instance-level supervision. Extensive experiments on the nuScenes benchmark demonstrate that Genesis achieves state-of-the-art performance across video and LiDAR metrics (FVD 16.95, FID 4.24, Chamfer 0.611), and benefits downstream tasks including segmentation and 3D detection, validating the semantic fidelity and practical utility of the generated data.

Overview

<div align="center"> <img src="assets/images/framework.png" width="1000"> </div>

News

[2025/09/18] Genesis is accepted by NeurIPS 2025🎉🎉🎉!

[2025/06/18] ArXiv paper release. Models/Code are coming soon. Please stay tuned! ☕️

Updates

• [x] Release Paper
• [ ] Release Full Models
• [ ] Release Inference Framework
• [ ] Release Training Framework

Citation

If you find Genesis is useful in your research or applications, please consider giving us a star 🌟 and citing it by the following BibTeX entry.

@article{guo2025genesis,
  title={Genesis: Multimodal Driving Scene Generation with Spatio-Temporal and Cross-Modal Consistency},
  author={Guo, Xiangyu and Wu, Zhanqian and Xiong, Kaixin and Xu, Ziyang and Zhou, Lijun and Xu, Gangwei and Xu, Shaoqing and Sun, Haiyang and Wang, Bing and Chen, Guang and others},
  journal={arXiv preprint arXiv:2506.07497},
  year={2025}
}