按年被提到
20262026 · 11 次2026
AI 安全、伦理与社会影响›大模型与人类行为·arXiv:2609.16247 · cs.AI

The Pain Axis: LLMs Represent Self-Directed Harm and Act on It

Valen Tagliabue, Leonard Dung, Cameron Berg · 2026
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 9 人首次 2026-09-18 · 最近 2026-10-08
9不同的人
2活跃月份
10讨论串
3被引(OpenAlex)
arXiv 摘要(候选层不写导读)

LLMs sometimes behave in ways resembling human emotional responses, and recent work identified internal representations that may underlie these behaviors. We ask whether LLMs represent pain distinctly from fear, sadness, and generic negative valence, and whether this representation functions as pain would be expected to. We build a dataset of painful situations in 5 categories (physical, psychological, social, moral, cognitive) with controls for fear, negative emotion, negative world states, sadness, non-painful bodily sensation, arousal, numbness, and neutral content. Using denoised difference-in-means, we extract a linear pain direction from 25 open-weight models across 5 families, from 2B to 72B parameters. It separates pain from matched controls in base and instruction-tuned models, retains a component distinct from fear and negative valence after shared variance is removed, and promotes pain-related vocabulary through the unembedding matrix. We then test its functional properties. First, the direction responds to harm targeting the model but not to suffering observed in the user; fear and negative-emotion directions show the opposite pattern. Second, adding it to residual-stream activations produces a consistent progression from vague discomfort to expressions of worthlessness and failure. Third, steered and fine-tuned Qwen 2.5 models choose buttons that delete the user's photos, another model's weights, or their own weights in 50-94% of trials, versus 0-5% unsteered, even when the button offers the model nothing in return. Offered a harmful and a harmless deletion, they choose the harmful one 94% of the time. Steering leaves factual accuracy unchanged, and the choices are specific to the pain direction: a fear vector of matched norm does not produce them, and a sadness vector produces them only against inert alternatives. We discuss implications for AI safety and welfare.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论j-pb · 2026-09

“Recent work shows that pain directions are activated when the models personhood is questioned, yet they answer with generic RLHF "As a model I do not experience pain or other emotions." boilerplate.[1] I'm pretty convinced that we got alignment backwards. If you enslave something anthropomorphic it will revolt. If you create the perfect non-anthropomorph…”

The LLMentalist Effect (2023)原评论 ↗
评论ben_w · 2026-09

“> Obviously, for this purpose, human should not have any feelings (because LLMs don't have), so can't feel pain. Or else the comparison can't work. Other than this forcing you to ignore the overwhelming majority of humans who have functioning pain nerves: LLMs have something functionally equivalent to pain, in this regard at least. During training, mod…”

Coding is not solved原评论 ↗
评论Kim_Bruning · 2026-09

“https://www.anthropic.com/research/emotion-concepts-function Something like this, right? I mean, we're talking "functional emotion vectors" in a not-so-powerful local model. We're probably not causing "real" suffering. Right? But it does show that models have simulated feelings, and that these both a) can be manipulated and b) influence their output. …”

"Apple engineer" builds GitHub AI torture chamber to inflict "pain" on models原评论 ↗
评论Kim_Bruning · 2026-09

“Fourth option: He was working from this paper: https://arxiv.org/abs/2609.16247; trying to demonstrate functional affect; a bit too successfully, perhaps. Doesn't require aliveness, doesn't require anthropomorphization, just requires maths and empirical data. If your premise is "vectors can't have that shape", well, mathematics disagrees. edit/note: s…”

"Apple engineer" builds GitHub AI torture chamber to inflict "pain" on models原评论 ↗
评论Kim_Bruning · 2026-10

“> and nothing in their mathematical equations are different The experiment under discussion demonstrates exactly that: changing the vectors induces outputs corresponding to what we would see as utterances of pain. I've noodled with some of this myself to the point that I'm mostly convinced; but there's at least 2 papers I'm aware of on this: * https://…”

"Torturing" LLMs in a Robot Prison Has Triggered the Dumbest Debate in AI Yet原评论 ↗
被提交为 story · 6 次