按年被提到
20192019 · 1 次2026
大模型与语言模型›量化、解码与分词·arXiv:1904.09324 · cs.CL

Mask-Predict: Parallel Decoding of Conditional Masked Language Models

Marjan Ghazvininejad, Omer Levy, Yinhan Liu, Luke Zettlemoyer · 2019
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2019-09-05 · 最近 2025-10-20
2不同的人
2活跃月份
2讨论串
498被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Most machine translation systems generate text autoregressively from left to right. We, instead, use a masked language modeling objective to train a model to predict any subset of the target words, conditioned on both the input text and a partially masked target translation. This approach allows for efficient iterative decoding, where we first predict all of the target words non-autoregressively, and then repeatedly mask out and regenerate the subset of words that the model is least confident about. By applying this strategy for a constant number of iterations, our model improves state-of-the-art performance levels for non-autoregressive and parallel decoding translation models by over 4 BLEU on average. It is also able to reach within about 1 BLEU point of a typical left-to-right transformer model, while decoding significantly faster.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论jaaustin · 2025-10

“To my knowledge this connection was first noted in 2021 in https://arxiv.org/abs/2107.03006 (page 5). We wanted to do text diffusion where you’d corrupt words to semantically similar words (like “quick brown fox” -> “speedy black dog”) but kept finding that masking was easier for the model to uncover. Historically this goes back even further to https://ar…”

BERT is just a single text diffusion step原评论 ↗
被提交为 story · 1 次