按年被提到
20162018 · 1 次2026
AI 安全、伦理与社会影响›智能体安全与滥用·arXiv:1604.00545 · cs.AI

The AGI Containment Problem

James Babcock, Janos Kramar, Roman Yampolskiy · 2016
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2018-05-15 · 最近 2021-01-12
2不同的人
2活跃月份
2讨论串
13被引(OpenAlex)
arXiv 摘要(候选层不写导读)

There is considerable uncertainty about what properties, capabilities and motivations future AGIs will have. In some plausible scenarios, AGIs may pose security risks arising from accidents and defects. In order to mitigate these risks, prudent early AGI research teams will perform significant testing on their creations before use. Unfortunately, if an AGI has human-level or greater intelligence, testing itself may not be safe; some natural AGI goal systems create emergent incentives for AGIs to tamper with their test environments, make copies of themselves on the internet, or convince developers and operators to do dangerous things. In this paper, we survey the AGI containment problem - the question of how to build a container in which tests can be conducted safely and reliably, even on AGIs with unknown motivations and capabilities that could be dangerous. We identify requirements for AGI containers, available mechanisms, and weaknesses that need to be addressed.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论logicallee · 2018-05

“No, I wasn't aware of that. This is a great link, thanks! To anyone else here is their excellent 2016 paper linked from the above: "Concrete Problems in AI Safety" https://arxiv.org/abs/1606.06565 and direct link to 29-page PDF: https://arxiv.org/pdf/1606.06565.pdf [PDF!] However, that PDF does not mention "AGI" even a single time, except in referen…”

OpenAI Research原评论 ↗
评论jimrandomh · 2021-01

“I'm the author of a previous paper about the AI containment problem. This new paper, Alfonseca et al, is kind of crap. This paper defines the term "containment problem" in a nonstandard way. This is different from how I use it in https://arxiv.org/pdf/1604.00545.pdf (which is largely about preventing side-channels and security vulnerabilities in realisti…”

Superintelligence cannot be contained: Lessons from Computability Theory原评论 ↗