“is trained to consult a calculator for arithmetic questions and consistently succeeds at it.”
LaMDA: Language Models for Dialog Applications
1370 亿参数的对话专用 Transformer,通过微调和外部工具提升安全与事实性
论文介绍 LaMDA:一族专用于对话的基于 Transformer 的神经语言模型,参数最多 1370 亿,在 1.56 万亿词的公开对话数据和网页文本上预训练。仅靠扩大规模难以改善安全性和事实依据,作者展示用标注数据微调,并让模型查询外部知识源(检索系统、翻译器、计算器),可显著改善这两个难题;他们用基于人类价值观的指标衡量安全性,用 groundedness 指标衡量事实性,并探索了 LaMDA 在教育和内容推荐领域的应用。
HN 上的人在讨论 LLM 是否有感知、能否可靠使用计算器等外部工具,以及训练成本时引用它,例如角色扮演珠穆朗玛峰的例子和 1024 块 TPUv3 预训练两个月。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“We can do whatever we want in vision, more or less [2].”
“The paper has an example of LaMDA thinking it is mount everest (see Table 4) when prompted to think that.”
“Note that, when told to be Mt. Everest, it pretends to be the mountain in a way that is just as convincing: LaMDA”
“as for your question, LaMDA states 2 months of 1024 TPUv3 for a pre-training, and TPUv3 isn't even the most recent TPU hardware”
“I guess it's probably something along the lines of "LaMDA: Language Models for Dialog Applications" (see LaMDA )”
看全部 10 条 →
“The three first authors are part of the Character.ai team.”
“Google’s model LaMDA has a system for detecting and avoiding false info, it’s interesting:”
“they were very clear in the LaMDA 2 paper that they only used public sources.”
“2022 LaMDA: LaMDA”
- LaMDA: Language Models for Dialog Applications ↗jonbaer · 2022-01-24 · 9 分