“Memory Augmented Large Language Models are Computationally Universal”
Memory Augmented Large Language Models are Computationally Universal
给 LLM 配上读写记忆,就能不改权重地模拟通用图灵机
论文证明 Transformer 类大语言模型配上外部读写记忆后具有计算通用性:只看有限长度输入的确定性语言模型等价于有限自动机,而加上读写记忆就可能处理任意大输入、模拟任意算法。作者用已有的 Flan-U-PaLM 540B 加联想式读写记忆,精确模拟了通用图灵机 U_{15,2},不需要修改模型权重,只靠设计一台由提示词编程的存储程序计算机。
评论者反复用它说明「带外部记忆的 LLM 在理论上图灵完备、能模拟特定图灵机」,并强调其意义在于上下文学习能走多远,而非方案本身是否实用。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“Also see this paper: Memory Augmented Large Language Models are Computationally Universal”
“when you add the external memory they (Flan-U-PaLM 540B to be specific) have been demonstrated to be capable of simulating a specific, well defined Turing machine without special training.”
“The significance of the paper is moreso the implications of how far ICL can take you rather than the ease/viability of the solution proposed.”
“Memory augmented LLMs are computationally universal - Memory Augmented Large Language Models are Computationally Universal”
“LLMs (at least the ones with read/write memory) can exactly simulate the execution of a universal Turing machine [1].”
看全部 8 条 →
“Transformers with memory can abstract recursively in theory. Memory Augmented Large Language Models are Computationally Universal.”
“But there were papers in 2023 with a different approach requiring external memory Memory Augmented Large Language Models are Computationally Universal too”
- Memory Augmented Large Language Models Are Computationally Universal ↗macawfish · 2023-02-21 · 1 分
- Memory Augmented Large Language Models Are Computationally Universal ↗macawfish · 2023-03-16 · 1 分
- Memory Augmented LLMs Are Computationally Universal ↗macawfish · 2023-03-17 · 1 分
- Memory Augmented Large Language Models Are Computationally Universal ↗gone35 · 2023-07-09 · 2 分