按年被提到
20182018 · 1 次2026
大模型与语言模型›词向量与预训练模型·arXiv:1810.04882 · cs.CL

Towards Understanding Linear Word Analogies

Kawin Ethayarajh, David Duvenaud, Graeme Hirst · 2018
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2018-11-02 · 最近 2023-04-29
2不同的人
2活跃月份
2讨论串
—被引(OpenAlex)
arXiv 摘要(候选层不写导读)

A surprising property of word vectors is that word analogies can often be solved with vector arithmetic. However, it is unclear why arithmetic operators correspond to non-linear embedding models such as skip-gram with negative sampling (SGNS). We provide a formal explanation of this phenomenon without making the strong assumptions that past theories have made about the vector space and word distribution. Our theory has several implications. Past work has conjectured that linear substructures exist in vector spaces because relations can be represented as ratios; we prove that this holds for SGNS. We provide novel justification for the addition of SGNS word vectors by showing that it automatically down-weights the more frequent word, as weighting schemes do ad hoc. Lastly, we offer an information theoretic interpretation of Euclidean distance in vector spaces, justifying its use in capturing word dissimilarity.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论ncraig · 2023-04

“This is an excellent question. I wonder if it's something like [1] on letter composition rather than meaning. [1] https://arxiv.org/pdf/1810.04882.pdf”

Launch Lamini: The LLM Engine for Rapidly Customizing Models as Good as ChatGPT原评论 ↗
被提交为 story · 1 次