按年被提到
20252025 · 12 次2026
AI 安全、伦理与社会影响›智能体安全与滥用·arXiv:2503.18813 · cs.CR

Defeating Prompt Injections by Design

从设计上击败提示注入
Edoardo Debenedetti, Ilia Shumailov, Tianqi Fan, Jamie Hayes, Nicholas Carlini, Daniel Fabian, Christoph Kern, Chongyang Shi, Andreas Terzis, Florian Tramèr · 2025

CaMeL 用控制流与数据流分离防御提示注入

开始阅读 ↗PDFalphaXiv 讨论
被推荐 · 13 人首次 2025-03-26 · 最近 2026-10-01
13不同的人
10活跃月份
18讨论串
—被引(OpenAlex)

针对 LLM agent 处理不可信数据时易受提示注入攻击的问题,提出 CaMeL:在 LLM 外围建立一层保护性系统,即使底层模型易受攻击也能保证安全。它显式从可信的用户查询中提取控制流和数据流,使 LLM 取回的不可信数据无法影响程序流程,并用能力(capability)机制在调用工具时执行安全策略,防止私有数据经未授权数据流外泄。在 AgentDojo 上,CaMeL 以可证明的安全性解决 77% 的任务,无防御系统为 84%。

HN 为什么提它

评论者认为它是目前最可信的提示注入解法之一:不依赖 AI 去检测攻击,而是靠架构上隔离不可信数据,常被当作 agent 安全设计的参考模型。

方法难度 3 / 5约 60 分钟经得起时间
读之前最好知道
1提示注入(prompt injection)
2LLM agent 与工具调用
3信息流控制与基于能力的安全(capability-based security)
4沙箱与最小权限原则

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
推荐simonw · 2025-04

“What changed is the new CaMeL paper from DeepMind, which notably does not rely on AI models to detect attacks:”

推荐 DeepMind 的 CaMeL 论文,它不依赖 AI 模型来检测攻击
Show HN: MCP-Shield – Detect security issues in MCP servers原评论 ↗
推荐groby_b · 2025-04

“CaMeL is rather interesting: Defeating Prompt Injections by Design”

觉得 CaMeL 很有意思,是面向 LLM 智能体的设计层面提示注入防御
The Policy Puppetry Attack: Novel bypass for major LLMs原评论 ↗
推荐rexpository · 2025-07

“CaMeL architecture is a good reference model: it surrounds the LLM with a capability-based”

推荐 CaMeL 架构为好的参考模型,用基于 capability 的机制包围 LLM
Supabase MCP can leak your entire SQL database原评论 ↗
推荐simonw · 2025-07

“what I consider a credible solution is the recent CaMeL paper from DeepMind:”

认为 DeepMind 的 CaMeL 论文是自己看好的可信方案
Code execution through email: How I used Claude to hack itself原评论 ↗
推荐msdz · 2026-03

“Research like [1] is what I love to discover”

称自己爱看这类研究:拆分决策与解析模型并用白名单工具,或可抵御注入
A GitHub Issue Title Compromised 4k Developer Machines原评论 ↗
引用simonw · 2025-08

“One of my blog posts was the foundation of the work described in the CaMeL paper from DeepMind for example:”

指出 DeepMind 的 CaMeL 工作以自己的一篇博客文章为基础
看全部 13 条 →
引用amonks · 2025-09

“the CaMeL paper. Defeating Prompt Injections by Design (2025): Defeating Prompt Injections by Design”

贴出 CaMeL 论文,从设计层面保护 LLM 智能体免受提示注入
Launch HN: Slashy (YC S25) – AI that connects to apps and does tasks原评论 ↗
引用brap · 2025-09

“Edit: found it. Defeating Prompt Injections by Design”

找到了所说的论文,其做法是让不可信的工具输出不进入主 LLM 的上下文
From MCP to shell: MCP auth flaws enable RCE in Claude Code, Gemini CLI and more原评论 ↗
引用phunterlau · 2026-01

“It reminded me of Google's CaMel work Defeating Prompt Injections by Design which introduces DSL to LLM prompt to prevent prompt injection by design”

联想到 Google 的 CaMeL,用 DSL 从设计上防止提示注入
Implementing a (Vibed) LLM Coding Agent in Prolog原评论 ↗
引用msdz · 2026-02

“Prompt injection is _probably_ solvable if something like [1] ever finds a mainstream implementation and adoption”

认为若这类设计被主流采用,提示注入很可能可以解决
Don't trust AI agents原评论 ↗
引用simonw · 2026-09

“makes me wonder if they've implemented ideas from the DeepMind CaMeL paper: Defeating Prompt Injections by Design”

怀疑对方是否采用了 CaMeL 的思路,把控制流与数据流分开来击败提示注入
Muse – Meta’s personal AI agent原评论 ↗
引用troelsSteegin · 2026-09

“CaMeL, from "Defeating Prompt Injections by Design", Defeating Prompt Injections by Design”

引用 CaMeL 这一设计模式,用来防御智能体的提示注入
What would a serious AI product look like?原评论 ↗
引用msdz · 2026-10

“Yes, and IIRC Google used this as part of a technique against prompt injection already [0], back when models were way more susceptible to it.”

指出 Google 早先已把它用于防御提示注入,当时模型更易受攻击
Is sandboxing sufficient to contain rogue agents?原评论 ↗
被提交为 story · 5 次