Oriel — AI & technology worth noticing, every day.

Token Pooling-Aware Training for Multi-Vector Embedding Models

Web page · https://huggingface.co/blog/token-pooling

First seen by Oriel on .

AI summary

This Hugging Face post presents pooling-aware training for multi-vector embedding models, preserving retrieval quality when Ward-clustered document token vectors are compressed to reduce storage.

  • HierarchicalTokenPooling merges similar token embeddings with Ward hierarchical clustering, represents each group by its normalized mean, and leaves query embeddings unchanged by default.
  • Pooling-aware training averages unpooled and compressed retrieval losses, using a straight-through estimator so encoder parameters learn which information must survive clustering.
  • With pool_factor=4, pooling-aware training reached 51.23 mean nDCG@10 versus 49.03 for ordinary training, a 2.20-point gain across NanoBEIR@13.
  • The trained example reduced document vectors from 58 to 16 while preserving the ranking of Mars, Jupiter, and Venus in a Red Planet query.
  • The implementation supports fixed maximums, proportional pool factors, calibrated adaptive thresholds, GPU kernels, and Sentence Transformers loss wrappers for custom deployment budgets.

#multi-vector retrieval #token pooling #sentence transformers #embedding models

Generated by gpt-5.6-sol. Check the original for details.

Aliases

SourceExternal IDURL
Hugging Face Bloghttps://huggingface.co/blog/token-poolinghttps://huggingface.co/blog/token-pooling Read discussion