按年被提到
20232023 · 1 次2026
大模型与语言模型›模型表达力与计算理论·arXiv:2301.13196 · cs.LG

Looped Transformers as Programmable Computers

Angeliki Giannou, Shashank Rajput, Jy-yong Sohn, Kangwook Lee, Jason D. Lee, Dimitris Papailiopoulos · 2023
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2023-06-15 · 最近 2026-09-10
2不同的人
2活跃月份
2讨论串
8被引(OpenAlex)
arXiv 摘要(候选层不写导读)

We present a framework for using transformer networks as universal computers by programming them with specific weights and placing them in a loop. Our input sequence acts as a punchcard, consisting of instructions and memory for data read/writes. We demonstrate that a constant number of encoder layers can emulate basic computing blocks, including embedding edit operations, non-linear functions, function calls, program counters, and conditional branches. Using these building blocks, we emulate a small instruction-set computer. This allows us to map iterative algorithms to programs that can be executed by a looped, 13-layer transformer. We show how this transformer, instructed by its input, can emulate a basic calculator, a basic linear algebra library, and in-context learning algorithms that employ backpropagation. Our work highlights the versatility of the attention mechanism, and demonstrates that even shallow transformers can execute full-fledged, general-purpose programs.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论nojvek · 2023-06

“As others have mentioned, turing machines can be achieved by only one instruction set. See https://en.wikipedia.org/wiki/One-instruction_set_computer Most popular being subleq (subract if less than or equal to zero). Subleq can be implemented with a transformer. See https://arxiv.org/pdf/2301.13196.pdf (Looped Transformers as Programmable Computers). …”

Attention is Turing complete [pdf]原评论 ↗
被提交为 story · 1 次