“They just look at some pretty images and interpret them to say that it does.”
Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation
在输入前加目标语言标记,单个模型即可多语互译并支持零样本翻译
论文提出用单个神经机器翻译(NMT)模型在多种语言之间翻译的简单方案:不改动基础系统的模型结构,只在输入句子开头加一个人工标记来指定目标语言,编码器、解码器和注意力都不变并在所有语言间共享,配合共享的 wordpiece 词表,参数量不增加。该方法常能在参数总量不变的情况下提升所有涉及语言对的翻译质量,在 WMT'14 和 WMT'15 上的若干语言对达到或超过当时最佳结果。模型还能对训练中从未直接见过的语言对做隐式桥接,表明神经翻译中迁移学习和零样本翻译是可行的,并有分析暗示模型内存在通用中间语表示。
评论者把它当作“单个共享模型可翻译训练中从未见过的语言对”(零样本翻译、隐式桥接)的出处,也有人质疑论文对通用中间语的论证只是靠观察可视化图像。
学习路径
读这篇之前,按顺序读书架上的 4 篇 · 参考文献来自 OpenAlex,顺序和要点由 Claude Code 排出先理解编码器-解码器与注意力的神经翻译系统,再看谷歌如何只加一个目标语言标记就实现多语言与零样本翻译。
- 1
编码器-解码器翻译框架的早期提出
读RNN 编码器-解码器的结构与短语表示
- 2
- 3
目标论文直接沿用的基础系统,架构完全不变,含 wordpiece
读8 层 LSTM 加注意力的结构与 wordpiece 分词
- 4
另一种多语言翻译思路:字符级无显式分词
读字符级建模如何跨语言共享,对比共享 wordpiece 词表
- 5Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation
目标论文:加目标语言标记的多语言翻译
读目标语言标记的做法,以及零样本翻译的结果
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“Just this week a new paper[1] was released using the GNMT architecture which teaches a machine translation system how to translate between language pairs it has never been taught before by using bridging from other language pair knowledge.”
“Deep learning is part of enormous advances in NLP[0], just as it has set records to accuracy in almost every field of machine perception, from vision to audio.”
“Google published a paper that explains exactly what they are doing and provides metrics as well [1,2].”
“their neural translation system has state-of-the-art performance for both English->German and German->English at least[1].”
“I admittedly understand very little about ML and how Google is using it in Translate, but this seems relevant.”
看全部 7 条 →
“This very misguided article seems to be a complete misunderstanding of two papers: [OTHER_PAPER] and Google's Multilingual Neural Machine Translation System”
- Zero shot translation learning ↗wsxiaoys · 2016-11-15 · 15 分
- Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation ↗somerandomness · 2016-11-15 · 1 分
- Multilingual Neural Machine Translation ↗cbennett · 2016-11-24 · 5 分
- Google's Zero-Shot Multilingual Neural Machine Translation System ↗gwern · 2017-02-24 · 2 分