引用gandalfgeek · 2023-01
“I was surprised to see the omission of a prompt technique called program-aided prompting.”
指出文章漏掉了程序辅助提示这一技巧,即让 LLM 写代码而非直接给答案
让大模型把问题写成程序,交给Python解释器去算
PAL 让大语言模型读懂自然语言问题后只负责生成程序形式的中间推理步骤,具体计算交给 Python 解释器执行,避免模型自己算错。在 BIG-Bench Hard 等 13 个数学、符号和算法推理任务上,这种做法比模型自己写推理链更准;用 Codex 的 PAL 在 GSM8K 上比使用 chain-of-thought 的 PaLM-540B 绝对高出约 15%。
HN 上的人把它当作“让模型写代码而不是直接给答案”的代表性做法,用来说明工具调用或程序辅助能弥补模型的计算错误。
“I was surprised to see the omission of a prompt technique called program-aided prompting.”
“PAL: Program-aided Language Models: PAL”
“Teaching LLMs to use tools isn't novel [3] [4], but it's a relatively recent idea.”
“We have ablations in PAL showing that both are indeed useful”