按年被提到
20242025 · 8 次2026
大模型与语言模型›大模型推理与思维链·arXiv:2412.06769 · cs.CL

Training Large Language Models to Reason in a Continuous Latent Space

训练大语言模型在连续潜空间中推理
Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, Yuandong Tian · 2024

Coconut:把最后隐藏状态直接回喂作为连续思维,在潜空间中推理

开始阅读 ↗PDFalphaXiv 讨论
被推荐 · 19 人首次 2024-12-10 · 最近 2026-09-06
19不同的人
11活跃月份
18讨论串
143被引(OpenAlex)

大语言模型通常在语言空间中通过思维链(CoT)推理,但语言空间未必最适合推理:多数词 token 只维持文本连贯,并非推理所必需。作者提出 Coconut(连续思维链):把模型最后的隐藏状态作为推理状态,称为「连续思维」,不解码成词,而是直接作为下一步的输入嵌入回馈给模型。这种潜空间推理能编码多个备选的下一步,使模型能做类似广度优先搜索的推理,而不是像 CoT 那样过早确定单一路径;在需要大量搜索规划的逻辑推理任务上,Coconut 优于 CoT,并在准确率与效率之间取得更好的权衡。

HN 为什么提它

每当有人谈到让模型在潜空间里推理而不是用英文 token 思考(neuralese)时,评论者就拿 Coconut 作为 Meta 的代表性研究来引用。

方法难度 4 / 5约 40 分钟经得起时间
读之前最好知道
1大语言模型与 Transformer 隐藏状态
2思维链提示(chain-of-thought)
3词嵌入与自回归解码
4图搜索与广度优先搜索(BFS)

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
推荐exhaze · 2024-12

“I think the wildest thing is actually Meta’s latest paper where they show a method for LLMs reasoning not in English, but in latent space”

称最离奇的是 Meta 这篇论文:让 LLM 不用英语而在潜空间中推理
GPT-5 is behind schedule原评论 ↗
推荐entrope · 2026-08

“I think Training Large Language Models to Reason in a Continuous Latent Space is a good description of the premise of reasoning in latent space, although [OTHER_PAPER] argues it's already what really happens.”

推荐它作为对潜空间推理前提的很好描述,尽管另一篇认为这本就是实际发生的
Qwen 3.8 27B is excellent, but it defaults to overthinking things原评论 ↗
引用verdverm · 2024-12

“Large language models (LLMs) are restricted to reason in the “language space”, where they typically express the reasoning process with a chain-of-thought (CoT) to solve a complex reasoning problem.”

引其摘要:LLM 被限制在语言空间用思维链推理(Coconut)
Coconut from Meta lets language models think without words原评论 ↗
引用padolsey · 2024-12

“I think you mean this: Training Large Language Models to Reason in a Continuous Latent Space”

指出对方想说的应是这篇,在潜空间而非英文 token 里做思维链(Coconut)
OpenAI O3 breakthrough high score on ARC-AGI-PUB原评论 ↗
引用davidhowlett · 2024-12

“The official pdf for the paper is at Training Large Language Models to Reason in a Continuous Latent Space”

给出这篇 Coconut 论文的官方 PDF,纠正他人所引
Coconut by Meta AI – Better LLM Reasoning with Chain of Continuous Thought?原评论 ↗
引用avodonosov · 2025-01

“Explicit <eot> is only used in training.”

补充说明推理步骤被潜在思维取代,显式 <eot> 只在训练中使用
Coconut by Meta AI – Better LLM Reasoning with Chain of Continuous Thought?原评论 ↗
看全部 18 条 →
引用scribu · 2025-02

“Would be curious to know how this stacks up against Coconut [1] which also uses latent space for reasoning.”

好奇所讨论方法与同样在潜空间推理的 Coconut 相比如何
Efficient Reasoning with Hidden Thinking原评论 ↗
引用eightysixfour · 2025-02

“There is research into reasoning models which can spend extra compute in latent space instead of in token space: Training Large Language Models to Reason in a Continuous Latent Space”

指出已有研究让推理模型把额外计算花在潜空间而非 token 空间
Watch R1 "think" with animated chains of thought原评论 ↗
引用voxgen · 2025-04

“based on their recent research such as coconut[1]”

把 Coconut 当作 Meta 近期的研究例证
The Llama 4 herd原评论 ↗
引用woadwarrior01 · 2025-05

“That's essentially the core idea in Coconut[1][2], to keep the reasoning traces in a continuous space.”

指出这正是 Coconut 的核心思想:把推理轨迹保持在连续空间
Beyond Semantics: Unreasonable Effectiveness of Reasonless Intermediate Tokens原评论 ↗
引用vonneumannstan · 2025-07

“Researchers are already pushing in this direction:”

指出研究者已在朝让 LLM 在潜空间而非 token 中推理的方向推进
Chain of thought monitorability: A new and fragile opportunity for AI safety原评论 ↗
引用supriyo-biswas · 2025-07

“There’s the coconut paper by meta, which kind of does a similar thing by reasoning in the latent space instead of doing it in tokens: Training Large Language Models to Reason in a Continuous Latent Space”

指出 Meta 的 Coconut 做了类似的事:在潜空间而非 token 中推理
Chain of thought monitorability: A new and fragile opportunity for AI safety原评论 ↗
引用krackers · 2025-12

“Isn't this sort of similar to latent looping? E.g. [1].”

询问这是否类似潜在循环,并以它为例
Universal Reasoning Model (53.8% pass 1 ARC1 and 16.0% ARC 2)原评论 ↗
引用a-t-c-g · 2026-04

“natural language may provide part of the scaffolding for reasoning, but the capability itself seems to depend more on learned transformations over internal representations than on language alone”

引它支持:推理能力更依赖对内部表示的学习变换,而非仅靠语言
TinyLoRA – Learning to Reason in 13 Parameters原评论 ↗
引用magicalist · 2026-04

“the discussion of latent space reasoning dates back to 2022[2]”

把它列为来源之一,指出潜空间推理的讨论可追溯到 2022 年
Claude Opus 4.7原评论 ↗
引用TeMPOraL · 2026-06

“They link to a Meta paper from 2024/2025 though: Training Large Language Models to Reason in a Continuous Latent Space.”

指出对方链接了 Meta 2024/2025 年的论文,即 neuralese 式连续潜空间推理
Claude Code's "extended thinking" is a summary- not authentic thinking原评论 ↗
引用supriyo-biswas · 2026-09

“I assume it's about reasoning in latent space e.g. the coconut paper[1]?”

推测讲的是潜空间推理,比如 Coconut 论文
OpenAI's new reasoning technique alarms AI safety experts原评论 ↗
引用tveita · 2026-09

“While the actual CoT in neuralese paper is Facebook's Coconut Training Large Language Models to Reason in a Continuous Latent Space - not sure if any production models use that one.”

指出 neuralese 思维链的实际论文是 Facebook 的 Coconut,不确定生产模型是否采用
What is nueralese and why is it bad原评论 ↗
被提交为 story · 2 次