READMem

<span style="color:orange;">

READMem: Robust Embedding Association for a Diverse Memory in Unconstrained Video Object Segmentation

</span>

by Stéphane Vujasinović, Sebastian Bullinger, Stefan Becker, Norbert Scherer-Negenborn, Michael Arens and Rainer Stiefelhagen

TL;DR: We manage the memory of STM like sVOS methods to better deal with long video. To attain long-term performance we estimate the inter-frame diversity of the base memory and integrate the embeddings of an incoming frame into the memory if it enhances the diversity. In return, we are able to limit the number of memory slots and deal with unconstrained video sequences without hindering the performance on short sequences and alleviate the need for a sampling interval.

[arXiv] - [BMVC Proceeding Paper]/[SUPP.] - [Video] - [Poster] - [BMVC Page]

<p align="center"> <img src="./docs/img/Qualitative_Results.png" width="95%"> </p>

---

News:

• Our poster was mentioned during the honorable mentions! 😄

📊 Some Quantitative Results

The following plots illustrate performance variations among sVOS baselines with and without our READMem extension on the LV1 dataset. The first plot showcases changes when varying the sampling interval $s_r$​, while the second depicts variations when adjusting the memory size $N$.

<p align="center"> <img src="./docs/img/Quantitative_Results_LV1.png" width="95%"> </p> <p align="center"> <img src="./docs/img/Quantitative_Results_LV1_bis.png" width="95%"> </p>

But check out our paper and supplementary material for more qualitative and quantitative results!

---

:books: Getting Started

The documentation is split in the following seperate markdown files:

:blue_book: Installation

:closed_book: Inference

:green_book: Evaluation

---

:black_nib: Citation

@inproceedings{Vujasinovic_2023_BMVC,
  author    = {Stephane Vujasinovic and Sebastian Bullinger and Stefan Becker and Norbert Scherer-Negenborn and Michael Arens and Rainer Stiefelhagen},
  title     = {READMem: Robust Embedding Association for a Diverse Memory in Unconstrained Video Object Segmentation},
  booktitle = {34th British Machine Vision Conference 2023, {BMVC} 2023, Aberdeen, UK, November 20-24, 2023},
  publisher = {BMVA},
  year      = {2023},
  url       = {https://papers.bmvc2023.org/0603.pdf}
}

---

:+1: Credits (Big thanks to those projects):

• MiVOS: GitHub - Paper
• STCN : GitHub - Paper
• QDMN : GitHub - Paper
• XMem: GitHub - Paper
• DAVIS: Webpage for the D17 dataset
• AFBURR: GitHub - Paper for the LV1 dataset
• DAVIS Toolkit: GitHub for the evaluation scripts