Zillow Indoor Dataset (ZInD)

The Zillow Indoor Dataset (ZInD) provides extensive visual data that covers a real world distribution of unfurnished residential homes. It consists of primary 360º panoramas with annotated room layouts, windows, doors and openings (W/D/O), merged rooms, secondary localized panoramas, and final 2D floor plans. The figure above illustrates the various representations (from left to right beyond capture): Room layout with W/D/O annotations, merged layouts, 3D textured mesh, and final 2D floor plan.

Definitions: Primary panoramas are those selected by annotators as having the "best" views of entire rooms, and are used to generate room layouts. The rest of the panoramas are secondary panoramas, which are provided for denser spatial data; they are localized within room layouts using a semi-automatic approach. An opening is an artificial construct that divides a large room into multiple parts. Note that openings are later processed for removal.

Paper

Zillow Indoor Dataset: Annotated Floor Plans With 360º Panoramas and 3D Room Layouts

Steve Cruz*, Will Hutchcroft*, Yuguang Li, Naji Khosravan, Ivaylo Boyadzhiev, Sing Bing Kang

Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021

[Paper] [Supplementary Material]

(* Equal contribution)

If you use the ZInD data or code please cite:

@inproceedings{ZInD,
  title     = {Zillow Indoor Dataset: Annotated Floor Plans With 360º Panoramas and 3D Room Layouts},
  author    = {Cruz, Steve and Hutchcroft, Will and Li, Yuguang and Khosravan, Naji and Boyadzhiev, Ivaylo and Kang, Sing Bing},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  month     = {June},
  year      = {2021},
  pages     = {2133--2143}
}

Data

Overview

ZInD is an RGB 360º panoramas dataset containing 67,448 panoramas taken in 1,575 unfurnished residential homes, annotated with 3D room layouts, 2D bounding boxes for W/D/O, merged room layouts, 3D camera poses, and final 2D floor plans.

The stats reported here are slightly different from those reported in the CVPR paper. If you need to cite the stats, please use the stats reported here.

Please refer to data organization for more details.

Capture Process

To capture home interiors at scale, we opted for sparse 360º panorama capture of every room in the home using an off-the-shelf 360º camera (such as Ricoh Theta V or Z1) paired with an iPhone. To do so, photographers across 20 US cities and 12 states were hired to do the capture; they were given specific instructions to ensure uniformity in the capture quality. Please see the FAQ section for more details.

Annotation Pipeline

The 3D floor plans were generated from a sparse set of RGB panoramas using our proprietary human-in-the-loop pipeline. Please see the FAQ section for more details.

Download

A one-home sample tour is given in sample_tour.

Registration for Download Request

If you are interested in downloading ZInD, please register an account on the Bridge Platform. During registration you will be required to agree to the Zillow Data Terms of Use. Before you take all these steps, however, we strongly suggest that you review the FAQ section first. Once your request has been approved, you will be notified. Please do not contact us unless it has been over two weeks since the request was made.

For non-commercial, academic institutions outside US and Canada, please select Other as a State/Province.

Get Server Token for ZInD Access

In Bridge Platform API, you can find the Server Token under API ACCESS Tab.

Setup/Install

Set up a conda environment:

conda create -n zind python=3.6
conda activate zind (Mac)
activate zind (Windows)

Install dependency libraries:

pip install -r requirements.txt

Batch Download

Use download_data.py to automatically download the data from the Bridge Platform, there is no need for you to interact with the API directly. Please note that the size of ZInD is about 40GB, so please make sure you have enough disk space before you start the download process.

python download_data.py -s <server_token> -o <output_folder>

CoVis Score Table

The CoVis scores used in the paper can be found here. The field names are self-explanatory.

Properties and Stats

Statistics for 1,575 homes and 67,448 panoramas. pri = primary, sec = secondary, “# annotator spaces” refers to spaces identified by annotators (which include closets and hallways), and “# rooms” refers to complete room layouts.

Feature | Total | Avg Per Home ------------ | ------------- | ------------- # panoramas (pri) | 33210 | 21.086 # panoramas (sec) | 34238 | 21.738 # floor plans | 2737 | 1.738 # annotator spaces | 29410 | 18.673 # rooms | 22484 | 14.276 # windows | 19403 | 12.319 # doors | 48759 | 30.958

Statistics on different room layout types. Since L-shaped layouts are common, we report that separately from others that are also Manhattan. Those that are non-Manhattan typically have room corner angles of 135 degree.

Layout Types | Cuboid | Manhattan-L | Manhattan-General | Non-Manhattan ------------ | ------------- | ------------- | ------------- | ------------- # Layouts | 11924 | 3273 | 2715 | 4572

Statistics on room layout count based on number of room corners

# Corners | 4 | 5 | 6 | 7 | 8 | 9 | 10+ ------------ | ------------- | ------------- | ------------- | ------------- | ------------- | ------------- | ------------- # Layouts | 11999 | 1002 | 3631 | 370 | 1416 | 154 | 3912

ZInD Partition

ZInD is partitioned to train/val/test (0.8 : 0.1 : 0.1). Those splits have similar distributions under the following metrics:

1. Layout complexities (cuboid, L-shape, etc.)
2. Number of floors
3. Number of primary panoramas
4. Number of secondary panoramas
5. Total area (to ensure that we have good balance between small/large homes)

The recommended train/val/test splits are in zind_partition.json

The partition script is available at code/partition_zind.py

Published HorizonNet Evaluation

Our dataset and training split has changed since the time of original submission. As such, the published evaluation numbers should be reproduced with the newly released train/test/val split for any future publications.

Visualization

Run the visualization script code/visualize_zind_cli.py

python code/visualize_zind_cli.py -i sample_tour -o <output_folder> --visualize-layout --visualize-floor-plan --raw --complete --visible --primary --secondary

More examples about how to run the visualization script can be found here

The visualization results for the sample tour are in render_data.

Structure3D Conversion & Rendering

Structured3D is a large-scale photo-realistic dataset containing 3.5K house designs proposed in a ECCV'20 paper. To extend the compatibility and potential applications of our ZInD dataset, we provided python scripts which convert original ZInD data format to Structure3D format with minimal modifications, and render the converted samples for visualization purposes. The modifications are due to a few extra components (e.g., windows, doors, and openings). In addition, we further provided the modified visualization scripts for visualizing the converted ZInD samples in an interactive manner.

A "s3d" branch is created for this update. Please see the scripts and details here.

License

The data is released under the ZInD Terms of Use, and the code is released under the Apache License.

Contact

ZInD@zillowgroup.com

Acknowledgement

We would like to thank the Zillow RMX engineering team for building the annotation infrastructure, and the ZO photographers for capturing the empty homes. We are grateful to Pierre Moulon and Lambert Wixson for discussions on ZInD. In addition, Ethan Wan has been very helpful in processing and validating data in preparation for release. Finally, the Bridge team has also been instrumental in making ZInD release a reality.

Frequently Asked Questions

• Can ZInD be downloaded free-of-charge?
• What are the conditions for using ZInD?
• How long do I wait after requesting for ZInD download?
• What are the important issues associated with the dataset that we need to be aware of?
• [What was the capture protocol