Awesome Multivector Retrieval

An extensive and commented list of resources on late-interaction multivector retrieval.

Contents

• Awesome Multivector Retrieval
  • Contents
  • Models
    • Foundational Models
    • General Models & Training
    • Compression & Token Pruning
    • Multimodal & Vision
    • Theory & Analysis
  • Retrieval
    • Indexing & Search Algorithms
    • Scoring Kernels
  • Software Libraries
    • Training & Inference Frameworks
    • Retrieval Engines & Indexes
    • Scoring Kernels
  • Model Checkpoints
    • General-Purpose
    • Specialized / Domain
  • Datasets and Encodings
    • MS MARCO v1
    • LoTTE-pooled
  • Multimedia Resources

Models

Foundational Models

• ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT<br> Omar Khattab, Matei Zaharia<br> SIGIR, 2020<br> 📄 paper | 🛠️ code

• COIL: Revisit Exact Lexical Match in Information Retrieval with Contextualized Inverted List<br> Luyu Gao, Zhuyun Dai, Jamie Callan<br> NAACL, 2021<br> 📄 paper

• ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction<br> Keshav Santhanam, Omar Khattab, Jon Saad-Falcon, Christopher Potts, Matei Zaharia<br> NAACL, 2022<br> 📄 paper | 🛠️ code

General Models & Training

• PyLate: Flexible Training and Retrieval for Late Interaction Models<br> Antoine Chaffin, Raphaël Sourty<br> CIKM, 2025<br> 📄 paper | 🛠️ code

• ColBERT-Zero: To Pre-train Or Not To Pre-train ColBERT models<br> Antoine Chaffin, Luca Arnaboldi, Amélie Chatelain, Florent Krzakala<br> arXiv, 2026<br> 📄 paper

• A Replicability Study of XTR<br> Rohan Jha, Reno Kriz, Benjamin Van Durme<br> arXiv, 2026<br> 📄 paper

• Your Embedding Model is SMARTer Than You Think<br> Jianrui Zhang, Hyun Jung Lee, Sukanta Ganguly, Tae-Eui Kam, Donghyun Kim, Yong Jae Lee<br> arXiv, 2026<br> 📄 paper | 🛠️ code

• Party is over: regularizing ColBERT models to fix efficient ANN methods<br> LightOn AI<br> Blog, 2026<br> 📝 blog

• NumColBERT: Non-Intrusive Numeracy Injection for Late-Interaction Retrieval Models<br> Haruki Fujimaki, Makoto P. Kato<br> arXiv, 2026<br> 📄 paper

• mDenseOn with the mLateOn: Open Multilingual, Long-Context, and Code Retrieval Models<br> LightOn AI<br> Blog, 2026<br> 📝 blog

• Jina-ColBERT-v2: A General-Purpose Multilingual Late Interaction Retriever<br> Rohan Jha, Bo Wang, Michael Günther, Georgios Mastrapas, Saba Sturua, Isabelle Mohr, Andreas Koukounas, Mohammad Kalim Akram, Nan Wang, Han Xiao<br> MRL Workshop, 2024<br> 📄 paper

Compression & Token Pruning

• Introducing Neural Bag of Whole-Words with ColBERTer: Contextualized Late Interactions using Enhanced Reduction<br> Sebastian Hofstatter, Omar Khattab, Sophia Althammer, Mete Sertkan, Allan Hanbury<br> CIKM, 2022<br> 📄 paper | 🛠️ code

• Joint Optimization of Multi-Vector Representation with Product Quantization<br> Yufan Fang, Jing Zhan, Yiqun Liu, Jiafeng Mao, Min Zhang, Shaoping Ma<br> NLPCC, 2022<br> 📄 paper

• Multi-Vector Retrieval as Sparse Alignment<br> Yujie Qian, Jinhyuk Lee, Sai Meher Karthik Duddu, Zhuyun Dai, Siddhartha Brahma, Iftekhar Naim, Tao Lei, Vincent Y. Zhao<br> arXiv, 2022<br> 📄 paper

• CITADEL: Conditional Token Interaction via Dynamic Lexical Routing for Efficient and Effective Multi-Vector Retrieval<br> Minghan Li, Sean C. Lin, Barlas Oguz, Arnab Ghoshal, Jimmy Lin, Yashar Mehdad, Wen-tau Yih, Xilun Chen<br> ACL, 2023<br> 📄 paper

• SLIM: Sparsified Late Interaction for Multi-Vector Retrieval with Inverted Indexes<br> Minghan Li, Sheng-Chieh Lin, Xueguang Ma, Jimmy Lin<br> SIGIR, 2023<br> 📄 paper

• Rethinking the Role of Token Retrieval in Multi-Vector Retrieval<br> Jinhyuk Lee, Zhuyun Dai, Sai Meher Karthik Duddu, Tao Lei, Iftekhar Naim, Ming-Wei Chang, Vincent Y. Zhao<br> NeurIPS, 2023<br> 📄 paper

• SPLATE: Sparse Late Interaction Retrieval<br> Thibault Formal, Stephane Clinchant, Herve Dejean, Carlos Lassance<br> SIGIR, 2024<br> 📄 paper

• Muvera: Multi-Vector Retrieval via Fixed Dimensional Encodings<br> Laxman Dhulipala, Majid Hadian, Rajesh Jayaram, Jason Lee, Vahab Mirrokni<br> NeurIPS, 2024<br> 📄 paper

• Enhancing ColBERT: A Method for Reducing Space Complexity and Accelerating Retrieval Speed<br> Hai Nguyen T., Huong Le T.<br> PACLIC, 2024<br> 📄 paper

• Token Pruning Optimization for Efficient Multi-vector Dense Retrieval<br> Shanxiu He, Mutasem Al-Darabsah, Suraj Nair, Jonathan May, Tarun Agarwal, Tao Yang, Choon Hui Teo<br> ECIR, 2025<br> 📄 paper

• CRISP: Clustering Multi-Vector Representations for Denoising and Pruning<br> João Veneroso, Rajesh Jayaram, Jinmeng Rao, Gustavo Hernández Ábrego, Majid Hadian, Daniel Cer<br> arXiv, 2025<br> 📄 paper

• Towards Lossless Token Pruning in Late-Interaction Retrieval Models<br> Yuxuan Zong, Benjamin Piwowarski<br> SIGIR, 2025<br> 📄 paper

• ColPruner: Combining Complementary Pruning Approaches for ColBERT in Web Search<br> Wondo Rhee, Chan Lim, Taewon Yoon, Gyuhyeon Choi, Jooyoung Lee<br> SIGIR Workshop, 2025<br> 📄 paper

• Sculpting the Vector Space: Towards Efficient Multi-Vector Visual Document Retrieval via Prune-then-Merge Framework<br> Yibo Yan, Mingdong Ou, Yi Cao, Xin Zou, Jiahao Huo, Shuliang Liu, James Kwok, Xuming Hu<br> arXiv, 2026<br> 📄 paper

• Multi-Vector Index Compression in Any Modality<br> Hanxiang Qin, Alexander Martin, Rohan Jha, Chunsheng Zuo, Reno Kriz, Benjamin Van Durme<br> SIGIR, 2026<br> 📄 paper | 🛠️ code

• A Brief Comparison of Training-Free Multi-Vector Sequence Compression Methods<br> Rohan Jha, Chunsheng Zuo, Reno Kriz, Benjamin Van Durme<br> ECIR (LIR Workshop), 2026<br> 📄 paper

• Learn to Pool: Lightweight Fine-Tuning for Flexible Multi-Vector Compression<br> Stefan Josef<br> ECIR (LIR Workshop), 2026<br> 📄 paper

• A Voronoi Cell Formulation for Principled Token Pruning in Late-Interaction Retrieval Models<br> Yash Kankanampati, Yuxuan Zong, Nadi Tomeh, Benjamin Piwowarski, Joseph Le Roux<br> SIGIR, 2026<br> 📄 paper

• Comparing Token Pruning Approaches for Multi-Vector Retrieval<br> Ferdinand Schlatt, Hanno Barschel, Matthias Hagen<br> SIGIR, 2026<br> 📄 paper

Multimodal & Vision

• ColPali: Efficient Document Retrieval with Vision Language Models<br> Manuel Faysse, Hugues Sibille, Tony Wu, Bilel Omrani, Gautier Viaud, Celine Hudelot, Pierre Colombo<br> ICLR, 2025<br> 📄 paper

• Video-ColBERT: Contextualized Late Interaction for Text-to-Video Retrieval<br> Arun V. Reddy, Alexander Martin, Eugene Yang, Andrew Yates, Kate Sanders, Kenton Murray, Reno Kriz, Celso M. de Melo, Benjamin Van Durme, Rama Chellappa<br> CVPR, 2025<br> 📄 paper

• ColMate: Contrastive Late Interaction and Masked Text for Multimodal Document Retrieval<br> Ahmed Masry, Megh Thakkar, Patrice Bechard, Sathwik Tejaswi Madhusudhan, Rabiul Awal, Shambhavi Mishra, Akshay Kalkunte Suresh, Srivatsava Daruru, Enamul Hoque, Spandana Gella, Torsten Scholak, Sai Rajeswar<br> EMNLP, 2025<br> 📄 paper

Theory & Analysis

• Multi-Vector Embeddings are Provably More Expressive than Single Vector Embeddings<br> Rajesh Jayaram<br> arXiv, 2026<br> 📄 paper

• Quantifying and Expanding the Theoretical Capacity of Late-Interaction Retrieval Models<br> Julian Killingback, Varad Ingale, Hamed Zamani, Cameron Musco<br> arXiv, 2026<br> 📄 paper

Retrieval

Indexing & Search Algorithms

• *Baleen: Robust Multi-Hop