“Locating and Editing Factual Associations in GPT shows an approach to editing a LLM to edit facts.”
Locating and Editing Factual Associations in GPT
用因果干预定位事实存储位置,并以秩一更新(ROME)改写事实
作者分析自回归 Transformer 如何存储和回忆事实关联,用因果干预找出决定事实预测的神经元激活,发现这些计算集中在处理主语 token 时的中间层前馈模块。据此提出秩一模型编辑(ROME),通过修改前馈权重来更新特定事实;在 zsRE 编辑任务上与现有方法相当,在新的反事实断言数据集上同时保持特异性与泛化。
评论者把它当作「直接编辑 LLM 中事实」的代表方法(ROME)来引用,并用它说明前馈层像键值存储,以及模型如何存储信息的假设与实验证据。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“The title made me think this was an approach that used memory editing techniques (e.g. ROME [1])”
“The ROME paper[1], among others, shows that the feed-forward layers function as a key-value store”
“I completely forgot to mention ROME in my last comment, where you can modify facts within an LLM Locating and Editing Factual Associations in GPT”
“Another paper ("ROME") Locating and Editing Factual Associations in GPT formulates hypothesis over how these models store information, and provide experimental evidence.”
“Sure, here are some citations:”