“There are some results in that paper, supposedly from exactly the technique described on an open dataset, that no one has ever been able to reproduce”
Distributed Representations of Sentences and Documents
Paragraph Vector:把句子和文档学成定长稠密向量
论文提出 Paragraph Vector,一种无监督算法,可把变长的句子、段落或文档学成定长稠密向量,训练目标是预测文档中的词。它克服了词袋特征丢失词序和语义的缺点,在多个文本分类和情感分析任务上取得了当时最好的结果。
评论者把它作为 paragraph embedding / doc2vec(gensim 的实现)的出处来引用,也有人指出其中部分报告结果一直无人能复现。
学习路径
读这篇之前,按顺序读书架上的 3 篇 · 参考文献来自 OpenAlex,顺序和要点由 Claude Code 排出这条路径帮我建立词向量(word2vec)及其前身的神经语言模型思路,理解 Paragraph Vector 如何把词向量方法推广到句子和文档。
- 1
统一的神经网络架构与词嵌入的早期实践,是词向量思路的前史。
读查表式的词嵌入层,以及用窗口上下文学习词表示的想法;不必深究各项 NLP 任务。
- 2
Paragraph Vector 的两个模型直接对应 CBOW 和 Skip-gram。
读CBOW 与 Skip-gram 的目标函数,以及词向量的类比性质。
- 3
目标论文沿用其中的层次 softmax 等训练技巧,并延伸其组合性的讨论。
读负采样与高频词降采样;理解训练为何变得快而好。
- 4Distributed Representations of Sentences and Documents
目标论文:给每个段落加一个向量,与词向量共同预测词。
读PV-DM 和 PV-DBOW 两种模型,以及推断新文档向量时固定其余参数的做法。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“Paragraph embedding was published this year as "Distributed Representations of Sentences and Documents". Distributed Representations of Sentences and Documents”
“This is about paragraph vectors: [OTHER_PAPER] Distributed Representations of Sentences and Documents”
“The gensim library now offers the 'Paragraph Vector' [1] approach to create vectors for longer ranges of text.”
“[1] Quoc V. Le, Tomas Mikolov, 2014, Distributed Representations of Sentences and Documents, Distributed Representations of Sentences and Documents”
“Doc2vec is the name of the gensim implementation of this paper [0].”
- Distributed Representations of Sentences and Documents ↗galapago · 2014-05-27 · 1 分
- Distributed Representations of Sentences and Documents (2014) [pdf] ↗espeed · 2016-12-12 · 68 分
- Distributed Representations of Sentences and Documents ↗sonabinu · 2019-01-10 · 5 分