按年被提到
20222023 · 1 次2026
强化学习与智能体›大模型智能体·arXiv:2211.03540 · cs.HC

Measuring Progress on Scalable Oversight for Large Language Models

Samuel R. Bowman, Jeeyoon Hyun, Ethan Perez, Edwin Chen, Craig Pettit, Scott Heiner, Kamilė Lukošiūtė, Amanda Askell, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini 等 46 人 · 2022
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2023-05-05 · 最近 2024-05-18
2不同的人
2活跃月份
2讨论串
91被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Developing safe and useful general-purpose AI systems will require us to make progress on scalable oversight: the problem of supervising systems that potentially outperform us on most skills relevant to the task at hand. Empirical work on this problem is not straightforward, since we do not yet have systems that broadly exceed our abilities. This paper discusses one of the major ways we think about this problem, with a focus on ways it can be studied empirically. We first present an experimental design centered on tasks for which human specialists succeed but unaided humans and current general AI systems fail. We then present a proof-of-concept experiment meant to demonstrate a key feature of this experimental design and show its viability with two question-answering tasks: MMLU and time-limited QuALITY. On these tasks, we find that human participants who interact with an unreliable large-language-model dialog assistant through chat -- a trivial baseline strategy for scalable oversight -- substantially outperform both the model alone and their own unaided performance. These results are an encouraging sign that scalable oversight will be tractable to study with present models and bolster recent findings that large language models can productively assist humans with difficult tasks.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论nmca · 2023-05

“It's important that humans are still able to understand the actions of these systems for safety reasons; the area focused on the problem you identify is "scalable oversight", see for example: https://arxiv.org/abs/2206.05802 https://arxiv.org/abs/2211.03540”

Slack GPT, the Future of AI in Slack原评论 ↗
评论agucova · 2024-05

“For context, the point of the Superalignment team was to work on a problem known as scalable oversight: the problem of aligning models in a way that holds up as models become more capable [1]. The reason behind this is that current alignment techniques (like RLHF), have limitations which are expected to worsen as models are scaled up [2]. This is to say,…”

OpenAI created a team to control 'superintelligent' AI – then let it wither原评论 ↗