按年被提到
20212022 · 1 次2026
大模型与语言模型›词向量与预训练模型·arXiv:2112.07577 · cs.CL

GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrieval

Kexin Wang, Nandan Thakur, Nils Reimers, Iryna Gurevych · 2021
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2022-03-03 · 最近 2023-09-08
2不同的人
2活跃月份
2讨论串
51被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Dense retrieval approaches can overcome the lexical gap and lead to significantly improved search results. However, they require large amounts of training data which is not available for most domains. As shown in previous work (Thakur et al., 2021b), the performance of dense retrievers severely degrades under a domain shift. This limits the usage of dense retrieval approaches to only a few domains with large training datasets. In this paper, we propose the novel unsupervised domain adaptation method Generative Pseudo Labeling (GPL), which combines a query generator with pseudo labeling from a cross-encoder. On six representative domain-specialized datasets, we find the proposed GPL can outperform an out-of-the-box state-of-the-art dense retrieval approach by up to 9.3 points nDCG@10. GPL requires less (unlabeled) data from the target domain and is more robust in its training than previous methods. We further investigate the role of six recent pre-training methods in the scenario of domain adaptation for retrieval tasks, where only three could yield improved results. The best approach, TSDAE (Wang et al., 2021) can be combined with GPL, yielding another average improvement of 1.4 points nDCG@10 across the six tasks. The code and the models are available at https://github.com/UKPLab/gpl.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论BoorishBears · 2023-09

“You just reinvented a 2 year old technique with a more expensive pipeline and missed performance gains (from the cross-encoder step): https://www.sbert.net/examples/domain_adaptation/README.html https://arxiv.org/abs/2112.07577”

LLMs, RAG, & the missing storage layer for AI原评论 ↗
被提交为 story · 1 次