引用svcrunch · 2022-11
“This includes late interaction [8], sparse encoding [9], and, most challengingly, dense encoding [10] systems.”
把后期交互作为神经搜索胜过 BM25 的例子之一
用残差压缩与去噪监督,让后期交互检索更准且索引缩小 6 到 10 倍
后期交互(late interaction)模型为每个 token 生成多向量表示,把相关性建模分解为可扩展的 token 级计算,效果更好,但占用空间大了一个数量级。论文提出 ColBERTv2,把激进的残差压缩机制与去噪监督策略结合,同时提升检索质量并降低空间占用。它在大量基准上取得域内外的当时最优质量,并把后期交互模型的空间占用缩小了 6 到 10 倍。
HN 评论者把它当作后期交互检索的标准参考,称赞 ColBERT / ColBERTv2 是泛化到新数据能力很强的检索方法,并用它向不熟悉者介绍 late interaction。
“This includes late interaction [8], sparse encoding [9], and, most challengingly, dense encoding [10] systems.”
“integrating ColBERT[2]/ColBERTv2[3], super strong retrieval methods, who are particularly good at generalising to new data”
“If you're not familiar with late interaction, here's the colBERTv2 paper: ColBERTv2”
“which looks to ease that burden”