按年被提到
20222024 · 2 次2026
大模型与语言模型›词向量与预训练模型·arXiv:2210.07316 · cs.CL

MTEB: Massive Text Embedding Benchmark

Niklas Muennighoff, Nouamane Tazi, Loïc Magne, Nils Reimers · 2022
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2024-03-12 · 最近 2024-11-05
2不同的人
2活跃月份
2讨论串
798被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Text embeddings are commonly evaluated on a small set of datasets from a single task not covering their possible applications to other tasks. It is unclear whether state-of-the-art embeddings on semantic textual similarity (STS) can be equally well applied to other tasks like clustering or reranking. This makes progress in the field difficult to track, as various models are constantly being proposed without proper evaluation. To solve this problem, we introduce the Massive Text Embedding Benchmark (MTEB). MTEB spans 8 embedding tasks covering a total of 58 datasets and 112 languages. Through the benchmarking of 33 models on MTEB, we establish the most comprehensive benchmark of text embeddings to date. We find that no particular text embedding method dominates across all tasks. This suggests that the field has yet to converge on a universal text embedding method and scale it up sufficiently to provide state-of-the-art results on all embedding tasks. MTEB comes with open-source code and a public leaderboard at https://github.com/embeddings-benchmark/mteb.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论yorwba · 2024-03

“Works for: Bitext mining: Given a sentence in one language, find its translation in a collection of sentences in another language using the cosine similarity of embeddings. Classification: identify the kind of text you're dealing with using logistic regression on the embeddings. Clustering: group similar texts together using k-means clustering on the e…”

Is Cosine-Similarity of Embeddings Really About Similarity?原评论 ↗
被提交为 story · 1 次