按年被提到
20242026 · 7 次2026
大模型与语言模型›模型表达力与计算理论·arXiv:2404.15758 · cs.CL

Let's Think Dot by Dot: Hidden Computation in Transformer Language Models

一点一点地想:Transformer 语言模型中的隐藏计算
Jacob Pfau, William Merrill, Samuel R. Bowman · 2024

无意义的填充 token 也能替代思维链,带来额外计算

开始阅读 ↗PDFalphaXiv 讨论
被提及 · 15 人首次 2024-04-27 · 最近 2026-10-06
15不同的人
12活跃月份
16讨论串
26被引(OpenAlex)

论文问思维链带来的提升,究竟来自类似人类的任务分解,还是来自额外 token 提供的更多计算。作者表明 Transformer 可以用无意义的填充 token(如“......”)代替思维链,解决两个没有中间 token 时解不了的困难算法任务,但学会使用填充 token 很困难,需要特定的密集监督才能收敛。论文还用一阶公式的量词深度刻画了填充 token 有用的问题类别,并指出这引发对 LLM 进行不可审计、与可见思维链脱节的隐藏计算的担忧。

HN 为什么提它

评论者把它当作“思维链不必有语义内容,填充 token 甚至只是点也能带来额外隐藏计算”的出处,用来讨论推理轨迹是否反映模型内部真实的计算。

实证研究难度 4 / 5约 45 分钟经得起时间
读之前最好知道
1思维链(chain of thought)
2Transformer 的计算表达能力
3一阶逻辑与量词深度(quantifier depth)
4序列模型的训练与监督方式

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
引用richdougherty · 2024-05

“Related? "Let's Think Dot by Dot: Hidden Computation in Transformer Language Models" Let's Think Dot by Dot”

贴出相关论文:Transformer 中填充 token 可以替代思维链
Vision Transformers Need Registers原评论 ↗
引用FeepingCreature · 2024-05

“let it even be dots”

提议哪怕只是点号,也是让模型隐藏额外计算的另一条路
The "Chain of Thought" Delusions原评论 ↗
引用zargon · 2024-05

“See for example, Let's Think Dot by Dot.”

举它为例:重复的填充计算 token 可与思维链效果相当
Ask HN: What is your ChatGPT customization prompt?原评论 ↗
引用FeepingCreature · 2024-05

“Let's Think Dot by Dot: Hidden Computation in Transformer Language Models Let's Think Dot by Dot”

引用它说明 Transformer 可用填充 token 做隐藏计算
Helen Keller on her life before self-consciousness (1908)原评论 ↗
引用mistercow · 2024-07

“which found that you can get many of the benefits of chain-of-thought simply by adding filler tokens”

指出该研究发现只加填充 token 就能获得思维链的许多好处
Show HN: Convert HTML DOM to semantic markdown for use in LLMs原评论 ↗
引用mistercow · 2025-01

“[1] Let's Think Dot by Dot: Hidden Computation in Transformer Language Models: Let's Think Dot by Dot”

把它列为“dot-dot-dotting”想法的出处:靠额外 token 做隐藏计算
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL原评论 ↗
看全部 16 条 →
引用gardnr · 2025-03

“There was a paper[1] from last year where the authors discovered getting the model to output anything during times of uncertainty, improved the generations overall.”

引去年论文:模型在不确定时输出任意内容,整体生成反而更好
Tencent's 'Hunyuan-T1'–The First Mamba-Powered Ultra-Large Model原评论 ↗
引用layoric · 2025-05

“"Let's Think Dot by Dot: Hidden Computation in Transformer Language Models" - Let's Think Dot by Dot”

列出这篇论文:填充点号能让 Transformer 做隐藏计算
引用krackers · 2025-07

“There are several such papers, off the top of my head one is Let's Think Dot by Dot”

举这篇为例:逐点填充 token 只对可并行化的问题有帮助
The End of Moore's Law for AI? Gemini Flash Offers a Warning原评论 ↗
引用Majromax · 2026-02

“a 2024 paper noted that simply adding dots to the output allowed some models to perform more latent computation resulting in higher-skill answers.”

引用它说明在输出里加点号,可让部分模型做更多潜在计算并给出更高水平的答案
Evaluating AGENTS.md: are they helpful for coding agents?原评论 ↗
引用prodigycorp · 2026-04

“Chain of thought does not need to be semantically useful to improve LLM performance. Let's Think Dot by Dot”

引用它说明思维链不必语义有用也能提升 LLM 表现
Talk like caveman原评论 ↗
引用Chance-Device · 2026-04

“I think these pretty much map out a little chronology of the research:”

把它列入研究脉络,说明中间 token 不必是有意义的文本
Talk like caveman原评论 ↗
引用neuroticnews25 · 2026-08

“But at least to some extent it's just model taking time thinking through the problem, and the trace produced doesn't have to be representative of what happens internally: Let's Think Dot by Dot”

引用它说明推理痕迹未必反映模型内部的真实计算
Qwen 3.8 27B is excellent, but it defaults to overthinking things原评论 ↗
引用dannyw · 2026-08

“even if you make thinking tokens literally just '....' (absolutely meaningless; zero information), you still see significant performance improvements: Let's Think Dot by Dot and [OTHER_PAPER] for some starters.”

指出就算思考 token 只是毫无信息的“....”,表现仍显著提升,并附上这篇
Qwen3.8-Flash-Next原评论 ↗
引用wren6991 · 2026-09

“I think this is a known phenomenon: even in non-reasoning models, adding useless/filler tokens before an answer improves task performance. The model is doing some computation during the filler. See: Let's Think Dot by Dot”

指出这是已知现象:答案前加无用填充 token 也能提升表现,模型在填充期间做计算
Claude discovers a novel enzyme system with CRISPR-like repeats原评论 ↗
引用thefxperson · 2026-10

“this seems supported by the abstract of the paper cited in the above article:”

指出文中引用的这篇论文摘要支持其说法:额外 token 带来的计算好处与 token 选择无关
I'm the AGI that's wiping out humanity原评论 ↗
被提交为 story · 1 次