按年被提到
20182018 · 1 次2026
大模型与语言模型›词向量与预训练模型·arXiv:1803.08493 · cs.CL

Contextual Salience for Fast and Accurate Sentence Vectors

Eric Zelikman, Richard Socher · 2018
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2018-04-28 · 最近 2020-08-25
2不同的人
2活跃月份
2讨论串
1被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Unsupervised vector representations of sentences or documents are a major building block for many language tasks such as sentiment classification. However, current methods are uninterpretable and slow or require large training datasets. Recent word vector-based proposals implicitly assume that distances in a word embedding space are equally important, regardless of context. We introduce contextual salience (CoSal), a measure of word importance that uses the distribution of context vectors to normalize distances and weights. CoSal relies on the insight that unusual word vectors disproportionately affect phrase vectors. A bag-of-words model with CoSal-based weights produces accurate unsupervised sentence or document representations for classification, requiring little computation to evaluate and only a single covariance calculation to ``train." CoSal supports small contexts, out-of context words and outperforms SkipThought on most benchmarks, beats tf-idf on all benchmarks, and is competitive with the unsupervised state-of-the-art.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论nwienert · 2020-08

“Look up tf-idf. We used this technique to build a similar product (now open sourced[0]) to Journal actually, building off an improved version of tf-idf called CoSaL[1]. Orbit is actually an insanely ambitious take on this problem space. The CoSaL algorithm is in there, it even runs using a forked version of Spotify’s C optimized algorithm. [0] https://g…”

Journal 2.0: Mental Space for What Matters原评论 ↗
被提交为 story · 1 次