“The most important paper in recent years is on CoT [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models and I'm not even sure what comes close.”
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
给出逐步推理示例,显著提升大模型推理表现
提出思维链(chain-of-thought)提示:在少样本示例里给出一系列中间推理步骤,让足够大的语言模型自然涌现出复杂推理能力。在三个大语言模型上,算术、常识和符号推理任务均有提升;论文称用 8 个思维链示例提示 540B 参数模型,在 GSM8K 数学应用题上达到当时最优,超过带验证器的微调 GPT-3。
它被反复当作原始出处引用,用来说明让模型先写出步骤就能得到更准确的答案,也被用来讨论 LLM 的推理能力、涌现现象和错误是否可纠正。
学习路径
读这篇之前,按顺序读书架上的 3 篇 · 参考文献来自 OpenAlex,顺序和要点由 Claude Code 排出这条路径建立 Transformer、语言模型缩放规律和少样本提示的背景,再读思维链提示如何在大模型上激发推理能力。
- 1
- 2
缩放定律说明模型规模带来的能力提升,对应思维链只在足够大的模型上才有效的结论。
读抓损失随参数量、数据、算力按幂律下降的主要结论。
- 3
- 4Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
给出逐步推理示例,显著提升大模型推理表现。
读抓思维链示例的构造、算术、常识与符号推理实验,以及模型规模带来的涌现。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“Better read: the original paper demonstrating that prompting chain-of-thought reasoning elicits superior results.”
“I can point you to the original chain of thought paper [0]”
“But clearly the technique was not invented by these authors.”
“Interesting that they used Chain of Thought Prompting[1] for improved reasoning so soon after its publication.”
“But they can be told to, and then the same network becomes much smarter: Chain-of-Thought Prompting Elicits Reasoning in Large Language Models”
看全部 20 条 →
“You're right for GPT 3, but it's an example of chain of thought reasoning, which seems to be a new area of research [1] and might get integrated into newer versions:”
“Or force the model to produce so called "chains-of-thought"[1] to make sure the model never leaves an instructional context...”
“no room for chain-of-thought, nothing in the prompt that it could append something to.”
“Btw, here's an example of how even a very simple zero-shot/prompt-engineering attempt to introduce a bit of system 2 reasoning into a language model can improve results.”
“If you ask it to write out the steps ahead of time you'll often get a more accurate answer [1]”
“Using chain of thought [slightly modified] example from Chain-of-Thought Prompting Elicits Reasoning in Large Language Models (ref'd in OP)”
“factuality can effectively be solved by chain-of-thought prompting[0]”
“Chain-of-Thought Prompting Elicits Reasoning in Large Language Models: Chain-of-Thought Prompting Elicits Reasoning in Large Language Models”
“It doesn't actually. See Chain-of-Thought Prompting Elicits Reasoning in Large Language Models”
“The problem is known for a while, for example this article [0] mentions it (as Chain of Thought reasoning).”
“You'll be wanting to find emergent stuff like this and not just crafting prompts to get the right response: Chain-of-Thought Prompting Elicits Reasoning in Large Language Models”
“One additional supporting evidence on the finite step reasoning hypothesis is that "train of thought" improves the performance of LLMs[1]”
“can really help some of the larger GPT models. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models”
“Just speculating, but this looks like precisely the sort of result one might expect from fine-tuning models on chain-of-thought-prompted interactions like those described by Wei et al. in 2022[1].”