“I think the wildest thing is actually Meta’s latest paper where they show a method for LLMs reasoning not in English, but in latent space”
Training Large Language Models to Reason in a Continuous Latent Space
Coconut:把最后隐藏状态直接回喂作为连续思维,在潜空间中推理
大语言模型通常在语言空间中通过思维链(CoT)推理,但语言空间未必最适合推理:多数词 token 只维持文本连贯,并非推理所必需。作者提出 Coconut(连续思维链):把模型最后的隐藏状态作为推理状态,称为「连续思维」,不解码成词,而是直接作为下一步的输入嵌入回馈给模型。这种潜空间推理能编码多个备选的下一步,使模型能做类似广度优先搜索的推理,而不是像 CoT 那样过早确定单一路径;在需要大量搜索规划的逻辑推理任务上,Coconut 优于 CoT,并在准确率与效率之间取得更好的权衡。
每当有人谈到让模型在潜空间里推理而不是用英文 token 思考(neuralese)时,评论者就拿 Coconut 作为 Meta 的代表性研究来引用。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“I think Training Large Language Models to Reason in a Continuous Latent Space is a good description of the premise of reasoning in latent space, although [OTHER_PAPER] argues it's already what really happens.”
“Large language models (LLMs) are restricted to reason in the “language space”, where they typically express the reasoning process with a chain-of-thought (CoT) to solve a complex reasoning problem.”
“I think you mean this: Training Large Language Models to Reason in a Continuous Latent Space”
“The official pdf for the paper is at Training Large Language Models to Reason in a Continuous Latent Space”
“Explicit <eot> is only used in training.”
看全部 18 条 →
“Would be curious to know how this stacks up against Coconut [1] which also uses latent space for reasoning.”
“There is research into reasoning models which can spend extra compute in latent space instead of in token space: Training Large Language Models to Reason in a Continuous Latent Space”
“based on their recent research such as coconut[1]”
“That's essentially the core idea in Coconut[1][2], to keep the reasoning traces in a continuous space.”
“Researchers are already pushing in this direction:”
“There’s the coconut paper by meta, which kind of does a similar thing by reasoning in the latent space instead of doing it in tokens: Training Large Language Models to Reason in a Continuous Latent Space”
“Isn't this sort of similar to latent looping? E.g. [1].”
“natural language may provide part of the scaffolding for reasoning, but the capability itself seems to depend more on learned transformations over internal representations than on language alone”
“the discussion of latent space reasoning dates back to 2022[2]”
“They link to a Meta paper from 2024/2025 though: Training Large Language Models to Reason in a Continuous Latent Space.”
“I assume it's about reasoning in latent space e.g. the coconut paper[1]?”
“While the actual CoT in neuralese paper is Facebook's Coconut Training Large Language Models to Reason in a Continuous Latent Space - not sure if any production models use that one.”
- Training LLMs to Reason in a Continuous Latent Space ↗omarsar · 2024-12-10 · 283 分
- Training Large Language Models to Reason in a Continuous Latent Space [pdf] ↗thunderbong · 2026-04-29 · 1 分