“Links to the relevant papers: Bag of Tricks for Efficient Text Classification: https://arxiv.org/abs/1607.01759v2 Enriching Word Vectors with Subword Information: https://arxiv.org/abs/1607.04606 Both fantastic papers. For those who aren't aware, Mikolov also helped create word2vec. One curious thing: this seems to use heirarchal softmax instead of th…”
Enriching Word Vectors with Subword Information
Continuous word representations, trained on large unlabeled corpora are useful for many natural language processing tasks. Popular models that learn such representations ignore the morphology of words, by assigning a distinct vector to each word. This is a limitation, especially for languages with large vocabularies and many rare words. In this paper, we propose a new approach based on the skipgram model, where each word is represented as a bag of character $n$-grams. A vector representation is associated to each character $n$-gram; words being represented as the sum of these representations. Our method is fast, allowing to train models on large corpora quickly and allows us to compute word representations for words that did not appear in the training data. We evaluate our word representations on nine different languages, both on word similarity and analogy tasks. By comparing to recently proposed morphological word representations, we show that our vectors achieve state-of-the-art performance on these tasks.
候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“This was posted in 2015, but now that fasttext (https://github.com/facebookresearch/fastText), just released by Facebook and can scale to Instagram-sized datasets, can create word vectors better than word2vec which account for context (https://arxiv.org/pdf/1607.04606v1.pdf), this type of analysis will only improve in the future.”
- Enriching Word Vectors with Subword Information [pdf] ↗putnam · 2017-07-04 · 1 分