AQATrack

The official implementation for the CVPR'2024 paper Autoregressive Queries for Adaptive Tracking with Spatio-Temporal Transformers

Models:[Models] Raw Results:[Raw Results] Training logs:[Training logs]

:sunny: Structure of AQATrack

:sunny: Highlights

:star2: New Autoregressive Query-based Tracking Framework

AQATrack is a simple, high-performance autoregressive query-based spatio-temporal tracker for adaptive learning the instantaneous target appearance changes in a sliding window fashion. Without any additional upadate strategy, AQATrack achieves SOTA performance on multiple benchmarks.

| Tracker | LaSOT (AUC)|LaSOT<sub>ext (AUC)|UAV123 (AUC)|TrackingNet (AUC)|TNL2K(AUC)|GOT-10K (AO) |:-----------:|:----------:|:-----------------:|:----------:|:---------------:|:--------:|:----------: | AQATrack-256| 71.4 | 51.2 | 70.7 | 83.8 | 57.8 | 73.8
| AQATrack-384| 72.7 | 52.7 | 71.2 | 84.8 | 59.3 | 76.0

Install the environment

Use the Anaconda

conda create -n aqatrack python=3.8
conda activate aqatrack
bash install.sh

Set project paths

Run the following command to set paths for this project

python tracking/create_default_local_file.py --workspace_dir . --data_dir ./data --save_dir ./output

After running this command, you can also modify paths by editing these two files

lib/train/admin/local.py  # paths about training
lib/test/evaluation/local.py  # paths about testing

Data Preparation

Put the tracking datasets in ./data. It should look like:

${PROJECT_ROOT}
 -- data
     -- lasot
         |-- airplane
         |-- basketball
         |-- bear
         ...
     -- got10k
         |-- test
         |-- train
         |-- val
     -- coco
         |-- annotations
         |-- images
     -- trackingnet
         |-- TRAIN_0
         |-- TRAIN_1
         ...
         |-- TRAIN_11
         |-- TEST

Training

Download pre-trained HiViT-Base weights and put it under $PROJECT_ROOT$/pretrained_models (see HiViT for more details).

bash train.sh

Test

python test_epoch.py

Evaluation

python tracking/analysis_results.py

Test FLOPs, and Speed

Note: The speeds reported in our paper were tested on a single RTX2080Ti GPU.

# Profiling AQATrack-ep150-full-256
python tracking/profile_model.py --script aqatrack --config AQATrack-ep150-full-256
# Profiling AQATrack-ep150-full-384
python tracking/profile_model.py --script aqatrack --config AQATrack-ep150-full-384

Acknowledgments

• Thanks for the EVPTrack and PyTracking library, which helps us to quickly implement our ideas.

Citation

If our work is useful for your research, please consider cite:

@inproceedings{xie2024autoregressive,
  title={Autoregressive Queries for Adaptive Tracking with Spatio-Temporal Transformers},
  author={Xie, Jinxia and Zhong, Bineng and Mo, Zhiyi and Zhang, Shengping and Shi, Liangtao and Song, Shuxiang and Ji, Rongrong},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={19300--19309},
  year={2024}
}