Auto-Regressively Generating Multi-View Consistent Images

<div align="center">

       

</div>

JiaKui Hu*, Yuxiao Yang*, Jialun Liu, Jinbo Wu, Chen Zhao, Yanye Lu <br>PKU, BaiduVis, THU<br>

🚀️🚀️ News:

• 2025-06-26: MV-AR is accepted by ICCV 2025 !!!
• 2025-07-26: The pretrained models of MV-AR are uploaded to huggingface and wisemodel.

Introduction

Diffusion-based multi-view image generation methods use a specific reference view for predicting subsequent views, which becomes problematic when overlap between the reference view and the predicted view is minimal, affecting image quality and multi-view consistency. Our MV-AR addresses this by using the preceding view with significant overlap for conditioning.

Updates

• [x] Rendered GSO test set.
• [x] Sampling codes for text-to-multi-view and image-to-multi-view.

TODO-lists:

• [ ] Sampling codes for text-image-to-multi-view and text-shape-to-multi-view.
• [ ] Training codes.

Results

Text to Multiview images

Image to Multiview images

Text + Geometric to Multiview images

Quick Start

Requirements

CUDA 12.4, Pytorch >= 2.4.0

pip install -r requirements.txt

Reproduce

1. Please download flan-t5-xl in ./pretrained_models;
2. Please download Cap3D_automated_Objaverse_full.csv in dataset/captions;
3. Please download models from huggingface or wisemodel, put them in ./pretrained_models;
4. Run:

# For t2mv on objaverse
sh sample_tcam2i.sh
# For t2mv on GSO
sh sample_icam2i_gso.sh
# For i2mv on GSO
sh sample_icam2i_gso.sh

The generated images will be saved to samples_objaverse_nv_ray/.

Train

Coming soon.

Acknowledgement

This repository is heavily based on LlamaGen. We would like to thank the authors of this work for publicly releasing their code.

For help or issues using this git, please feel free to submit a GitHub issue.

For other communications related to this git, please contact jkhu29@stu.pku.edu.cn.

Citation

@inproceedings{hu2025mvar,
  title={Auto-Regressively Generating Multi-View Consistent Images},
  author={Hu, JiaKui and Yang, Yuxiao and Liu, Jialun and Wu, Jinbo and Zhao, Chen and Lu, Yanye},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
  year={2025}
}