引用puppystench · 2026-04
“showing that models infer the source of text from how it sounds, not the actual role labels the harness would provide.”
引用它说明模型凭文本的语气而非角色标签判断来源
模型凭文字像谁来判断说话者,因此注入会奏效
论文把提示注入归因于角色混淆:模型是根据文字听起来像谁,而不是标注的角色标签来判断文本来源。作者设计角色探针测量模型内部如何感知「谁在说话」,发现被注入的文本与它模仿的可信角色占据相同的表示空间;并提出零样本攻击 CoT Forgery,在用户提示和工具输出中伪造推理,对前沿模型达到 60% 攻击成功率,而基线接近零。角色混淆的程度甚至能在生成第一个 token 之前就预测攻击是否成功。
评论者引用它说明模型靠文字风格而非角色标签推断文本来源,因此提示注入难以靠标签防住,也连带影响对隐藏推理的讨论。
“showing that models infer the source of text from how it sounds, not the actual role labels the harness would provide.”
“a known phenomenon: Prompt Injection as Role Confusion”
“People will keep hiding reasoning because it allows prompt injection Prompt Injection as Role Confusion”
“Which sounds promising... until you see results like this one: Prompt Injection as Role Confusion”