“they are bad at things like predicting what the output of some code will be: Show Your Work”
Show Your Work: Scratchpads for Intermediate Computation with Language Models
让语言模型把中间步骤写到草稿纸上,显著提升多步计算能力
大型预训练语言模型擅长一次完成的任务,但做整数加法、执行程序这类需要无界多步计算的任务就很吃力。论文发现,这些模型在被要求一步一步、展示中间计算结果时,即使在少样本设定下也能完成复杂的多步计算。作者让 Transformer 把中间步骤输出到「草稿纸(scratchpad)」里,在从长加法到执行任意程序等一系列日益复杂的任务上,草稿纸显著提升了语言模型的多步计算能力。
HN 评论者把它当作 chain-of-thought 一类做法的早期例证来引用:让模型把中间计算写出来,能大幅提升算术和预测代码输出的准确率,是后续研究时间线的第一步。
学习路径
读这篇之前,按顺序读书架上的 6 篇 · 参考文献来自 OpenAlex,顺序和要点由 Claude Code 排出这条路径帮你理解语言模型难以做多步计算的原因:先看 Transformer 与 GPT-3,再看让网络学会执行程序、循环计算的先行工作,最后读草稿纸方法。
- 1
- 2
- 3
- 4
- 5
- 6
- 7Show Your Work: Scratchpads for Intermediate Computation with Language Models
目标论文:把中间步骤写到草稿纸上,提升多步计算能力。
读草稿纸格式、加法与程序执行任务,以及少样本与微调的结果。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“allows them to do much better not just at arithmetic but also things like predicting the output of some code: Show Your Work”
“Here's one example from 2021 demonstrating training an LLM to use a scratchpad ("talking to itself") to greatly improve accuracy on arithmetic problems:”
“I understand scratchpads (e.g. [0] Show Your Work: Scratchpads for Intermediate Computation with Language Models)”
“I think these pretty much map out a little chronology of the research:”
- Show Your Work: Scratchpads For Intermediate Computation With Language Models ↗optimalsolver · 2022-07-18 · 2 分