按年被提到
20222023 · 2 次2026
AI 安全、伦理与社会影响›大模型与人类行为·arXiv:2212.09251 · cs.CL

Discovering Language Model Behaviors with Model-Written Evaluations

Ethan Perez, Sam Ringer, Kamilė Lukošiūtė, Karina Nguyen, Edwin Chen, Scott Heiner, Craig Pettit, Catherine Olsson, Sandipan Kundu, Saurav Kadavath, Andy Jones, Anna Chen 等 63 人 · 2022
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2023-02-15 · 最近 2023-04-07
2不同的人
2活跃月份
2讨论串
186被引(OpenAlex)
arXiv 摘要(候选层不写导读)

As language models (LMs) scale, they develop many novel behaviors, good and bad, exacerbating the need to evaluate how they behave. Prior work creates evaluations with crowdwork (which is time-consuming and expensive) or existing data sources (which are not always available). Here, we automatically generate evaluations with LMs. We explore approaches with varying amounts of human effort, from instructing LMs to write yes/no questions to making complex Winogender schemas with multiple stages of LM-based generation and filtering. Crowdworkers rate the examples as highly relevant and agree with 90-100% of labels, sometimes more so than corresponding human-written datasets. We generate 154 datasets and discover new cases of inverse scaling where LMs get worse with size. Larger LMs repeat back a dialog user's preferred answer ("sycophancy") and express greater desire to pursue concerning goals like resource acquisition and goal preservation. We also find some of the first examples of inverse scaling in RL from Human Feedback (RLHF), where more RLHF makes LMs worse. For example, RLHF makes LMs express stronger political views (on gun rights and immigration) and a greater desire to avoid shut down. Overall, LM-written evaluations are high-quality and let us quickly discover many novel LM behaviors.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论gwern · 2023-02

“I think there is a small possibility that optimal scaling as we understand it right now (scaling compute + task/data diversity + parameter count in an optimal fashion), could just emerge (or scale) consciousness. We at present do not know what leads to emergence in the many now-documented examples of emergent capabilities in large NNs (see https://arxiv.o…”

It Looks Like You’re Trying to Take over the World原评论 ↗
评论lcnPylGDnU4H9OF · 2023-04

“It sounds like this is unexpected behavior, even from the perspective of those developing at the lowest level in these models. From the essay: > What I find fascinating about this is that these extremely problematic behaviours are not the system working as intended: they are bugs! And we haven’t yet found a reliable way to fix them. Right below that is…”

We need to tell people ChatGPT will lie to them, not debate linguistics原评论 ↗