按年被提到
20162025 · 2 次2026
大模型与语言模型›序列模型与注意力机制·arXiv:1603.08983 · cs.NE

Adaptive Computation Time for Recurrent Neural Networks

Alex Graves · 2016
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2025-02-10 · 最近 2025-10-08
2不同的人
2活跃月份
2讨论串
335被引(OpenAlex)
arXiv 摘要(候选层不写导读)

This paper introduces Adaptive Computation Time (ACT), an algorithm that allows recurrent neural networks to learn how many computational steps to take between receiving an input and emitting an output. ACT requires minimal changes to the network architecture, is deterministic and differentiable, and does not add any noise to the parameter gradients. Experimental results are provided for four synthetic problems: determining the parity of binary vectors, applying binary logic operations, adding integers, and sorting real numbers. Overall, performance is dramatically improved by the use of ACT, which successfully adapts the number of computational steps to the requirements of the problem. We also present character-level language modelling results on the Hutter prize Wikipedia dataset. In this case ACT does not yield large gains in performance; however it does provide intriguing insight into the structure of the data, with more computation allocated to harder-to-predict transitions, such as spaces between words and ends of sentences. This suggests that ACT or other adaptive computation methods could provide a generic method for inferring segment boundaries in sequence data.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论atrudeau · 2025-10

“Very cool to see the effectiveness of recurrence on ARC. For those interested in recurrence, here are other works that leverage a similar approach for other types of problems: Language modeling: Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach https://arxiv.org/pdf/2502.05171 Puzzle solving: A Simple Loss Function for Con…”

Less is more: Recursive reasoning with tiny networks原评论 ↗
被提交为 story · 1 次