按年被提到
20222022 · 1 次2026
大模型与语言模型›词向量与预训练模型·arXiv:2202.08904 · cs.CL

SGPT: GPT Sentence Embeddings for Semantic Search

Niklas Muennighoff · 2022
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2022-02-22 · 最近 2023-08-15
2不同的人
2活跃月份
2讨论串
173被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Decoder transformers have continued increasing in scale reaching hundreds of billions of parameters. Due to their scale the same decoder sets state-of-the-art results on various language tasks via prompting or fine-tuning. Yet, these large foundation models remain unusable for the related fields of semantic search and sentence embeddings. This prevents possibly new state-of-the-art results and forces organizations to train and maintain separate models. To this end, we propose SGPT to use decoders for sentence embeddings and semantic search via prompting or fine-tuning. At 5.8 billion parameters SGPT improves on the previously best sentence embeddings by a margin of 7% and outperforms a concurrent method with 175 billion parameters as measured on the BEIR search benchmark. Code, models and result files are freely available at https://github.com/Muennighoff/sgpt.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论Palmik · 2023-08

“It seems unlikely that raw llama2 will perform better than purpose made encoder models like bge [1], gte [2], e5 [3] or instructor, despite it's much much larger size (for the tasks people usually need embeddings for). You can probably get it to behave well with a fine tuning like this: https://arxiv.org/pdf/2202.08904.pdf [1] https://huggingface.co/BAA…”

Show HN: Llama2 Embeddings FastAPI Server原评论 ↗
被提交为 story · 1 次