Point-Query Quadtree for Crowd Counting, Localization, and More (ICCV 2023)

This repository includes the official implementation of the paper:

Point-Query Quadtree for Crowd Counting, Localization, and More

International Conference on Computer Vision (ICCV), 2023

Chengxin Liu¹, Hao Lu¹, Zhiguo Cao¹, Tongliang Liu²

¹Huazhong University of Science and Technology, China

²The University of Sydney, Australia

[Paper] | [Supplementary]

Highlights

We formulate crowd counting as a decomposable point querying process, where sparse input points could split into four new points when necessary. This formulation exhibits many appealing properties:

• Intuitive: The input and output are both interpretable and steerable

• Generic: PET is applicable to a number of crowd-related tasks, by simply adjusting the input format

• Effective: PET reports state-of-the-art crowd counting and localization results

Installation

• Required packages:

torch
torchvision
numpy
opencv-python
scipy
matplotlib

• Install packages:

pip install -r requirements.txt

Data Preparation

• Download crowd-counting datasets, e.g., ShanghaiTech.

• We expect the directory structure to be as follows:

PET
├── data
│    ├── ShanghaiTech
├── datasets
├── models
├── ...

• Alternatively, you can define the path of the dataset in datasets/init.py

• For UCF-QNRF, JHU-Crowd++, and NWPU-Crowd datasets, please refer to preprocess_dataset.py:

  • change dataset and data_root
  • run python preprocess_dataset.py

Training

• Download ImageNet pretrained vgg16_bn, and put it in pretrained folder. Or you can define your pre-trained model path in models/backbones/vgg.py

• To train PET on ShanghaiTech PartA, run

  sh train.sh
  

Evaluation

• Modify eval.sh
  • change --resume to your local model path
• Run

sh eval.sh

Pretrained Models

• Environment:

python==3.8
pytorch==1.12.1
torchvision==0.13.1

• Models:

| Dataset | Model Link | Training Log | MAE | | ------------------------ | ----------- | --------------| ----| | ShanghaiTech PartA | SHA_model.pth | SHA_log.txt | 49.08 | | ShanghaiTech PartB | SHB_model.pth | SHB_log.txt | 6.18 | | UCF_QNRF | UCF_QNRF.pth | - | - | | JHU_Crowd | JHU_Crowd.pth | - | - | | NWPU_Crowd | NWPU_Crowd.pth | - | - |

Frequently Asked Questions (FAQ)

• The model trained on my custom dataset does not perform well, why?

  • Please check the load_data function in your custom dataset. The input format should be (y, x) instead of (x, y). If the input annotations are wrong during training, the output of the trained model could be abnormal.

• How to deal with images with no person?

  • Please refer to this issue.

Citation

If you find this work helpful for your research, please consider citing:

@InProceedings{liu2023pet,
  title={Point-Query Quadtree for Crowd Counting, Localization, and More},
  author={Liu, Chengxin and Lu, Hao and Cao, Zhiguo and Liu, Tongliang},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
  year={2023}
}

Permission

This code is for academic purposes only. Contact: Chengxin Liu (cx_liu@hust.edu.cn)

Acknowledgement

We thank the authors of DETR and P2PNet for open-sourcing their work.