按年被提到
20162016 · 3 次2026
大模型与语言模型›序列模型与注意力机制·arXiv:1610.10099 · cs.CL

Neural Machine Translation in Linear Time

Nal Kalchbrenner, Lasse Espeholt, Karen Simonyan, Aaron van den Oord, Alex Graves, Koray Kavukcuoglu · 2016
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 4 人首次 2016-11-01 · 最近 2023-01-31
4不同的人
2活跃月份
4讨论串
414被引(OpenAlex)
arXiv 摘要(候选层不写导读)

We present a novel neural network for processing sequences. The ByteNet is a one-dimensional convolutional neural network that is composed of two parts, one to encode the source sequence and the other to decode the target sequence. The two network parts are connected by stacking the decoder on top of the encoder and preserving the temporal resolution of the sequences. To address the differing lengths of the source and the target, we introduce an efficient mechanism by which the decoder is dynamically unfolded over the representation of the encoder. The ByteNet uses dilation in the convolutional layers to increase its receptive field. The resulting network has two core properties: it runs in time that is linear in the length of the sequences and it sidesteps the need for excessive memorization. The ByteNet decoder attains state-of-the-art performance on character-level language modelling and outperforms the previous best results obtained with recurrent networks. The ByteNet also achieves state-of-the-art performance on character-to-character machine translation on the English-to-German WMT translation task, surpassing comparable neural translation models that are based on recurrent networks with attentional pooling and run in quadratic time. We find that the latent alignment structure contained in the representations reflects the expected alignment between the tokens.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论dharma1 · 2016-11

“Really like what you're doing with SpaCy and explosionAI, good stuff:) What do you think about dilated convolutional encoder/decoder networks [1]? Useful for NLP beyond machine translation? [1] https://arxiv.org/abs/1610.10099, https://github.com/paarthneekhara/byteNet-tensorflow”

Embed, encode, attend, predict: the new deep learning formula for NLP models原评论 ↗
评论talolard · 2023-01

“I'm a native Hebrew speaker, where we have wild word morphology, and also worked on NLP at a large bank where we trained models to anlyze Bloomberg chats (that are almost english but behave like a DSL). In both cases, "token-free" has been the way to go for extractive tasks like entity recognition. It's unfortunately a technique that is off the beaten p…”

Towards A Token-Free Future In NLP (2022)原评论 ↗
被提交为 story · 2 次