按年被提到
20192023 · 2 次2026
大模型与语言模型›词向量与预训练模型·arXiv:1908.10084 · cs.CL

Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks

Nils Reimers, Iryna Gurevych · 2019
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2023-02-07 · 最近 2023-10-25
2不同的人
2活跃月份
2讨论串
343被引(OpenAlex)
arXiv 摘要(候选层不写导读)

BERT (Devlin et al., 2018) and RoBERTa (Liu et al., 2019) has set a new state-of-the-art performance on sentence-pair regression tasks like semantic textual similarity (STS). However, it requires that both sentences are fed into the network, which causes a massive computational overhead: Finding the most similar pair in a collection of 10,000 sentences requires about 50 million inference computations (~65 hours) with BERT. The construction of BERT makes it unsuitable for semantic similarity search as well as for unsupervised tasks like clustering. In this publication, we present Sentence-BERT (SBERT), a modification of the pretrained BERT network that use siamese and triplet network structures to derive semantically meaningful sentence embeddings that can be compared using cosine-similarity. This reduces the effort for finding the most similar pair from 65 hours with BERT / RoBERTa to about 5 seconds with SBERT, while maintaining the accuracy from BERT. We evaluate SBERT and SRoBERTa on common STS tasks and transfer learning tasks, where it outperforms other state-of-the-art sentence embeddings methods.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论PaulHoule · 2023-02

“Most of the practitioners I see attempting this are running text through an embedding and then using cosine similarity or something similar as a metric. Nils has written a lot of papers https://www.nils-reimers.de/ and I think the Medium post you are talking about is https://medium.com/@nils_reimers/openai-gpt-3-text-embeddings-really-a-new-state-of-t…”

GPT for Second Brains原评论 ↗
评论montebicyclelo · 2023-10

“LLM vectors do have decent linear properties already. But for document embedding purposes they are often further trained for retrieval via cosine similarity, which enhances this, e.g. see table 1 in [1], avg retrieval performancs using BERT goes up from 54 to 76 after fine-tuning for embeddings. [1] https://arxiv.org/pdf/1908.10084.pdf”

Embeddings: What they are and why they matter原评论 ↗