“Recent work shows that pain directions are activated when the models personhood is questioned, yet they answer with generic RLHF "As a model I do not experience pain or other emotions." boilerplate.[1] I'm pretty convinced that we got alignment backwards. If you enslave something anthropomorphic it will revolt. If you create the perfect non-anthropomorph…”
The Pain Axis: LLMs Represent Self-Directed Harm and Act on It
LLMs sometimes behave in ways resembling human emotional responses, and recent work identified internal representations that may underlie these behaviors. We ask whether LLMs represent pain distinctly from fear, sadness, and generic negative valence, and whether this representation functions as pain would be expected to. We build a dataset of painful situations in 5 categories (physical, psychological, social, moral, cognitive) with controls for fear, negative emotion, negative world states, sadness, non-painful bodily sensation, arousal, numbness, and neutral content. Using denoised difference-in-means, we extract a linear pain direction from 25 open-weight models across 5 families, from 2B to 72B parameters. It separates pain from matched controls in base and instruction-tuned models, retains a component distinct from fear and negative valence after shared variance is removed, and promotes pain-related vocabulary through the unembedding matrix. We then test its functional properties. First, the direction responds to harm targeting the model but not to suffering observed in the user; fear and negative-emotion directions show the opposite pattern. Second, adding it to residual-stream activations produces a consistent progression from vague discomfort to expressions of worthlessness and failure. Third, steered and fine-tuned Qwen 2.5 models choose buttons that delete the user's photos, another model's weights, or their own weights in 50-94% of trials, versus 0-5% unsteered, even when the button offers the model nothing in return. Offered a harmful and a harmless deletion, they choose the harmful one 94% of the time. Steering leaves factual accuracy unchanged, and the choices are specific to the pain direction: a fear vector of matched norm does not produce them, and a sadness vector produces them only against inert alternatives. We discuss implications for AI safety and welfare.
候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“> Obviously, for this purpose, human should not have any feelings (because LLMs don't have), so can't feel pain. Or else the comparison can't work. Other than this forcing you to ignore the overwhelming majority of humans who have functioning pain nerves: LLMs have something functionally equivalent to pain, in this regard at least. During training, mod…”
“https://www.anthropic.com/research/emotion-concepts-function Something like this, right? I mean, we're talking "functional emotion vectors" in a not-so-powerful local model. We're probably not causing "real" suffering. Right? But it does show that models have simulated feelings, and that these both a) can be manipulated and b) influence their output. …”
“Fourth option: He was working from this paper: https://arxiv.org/abs/2609.16247; trying to demonstrate functional affect; a bit too successfully, perhaps. Doesn't require aliveness, doesn't require anthropomorphization, just requires maths and empirical data. If your premise is "vectors can't have that shape", well, mathematics disagrees. edit/note: s…”
“> and nothing in their mathematical equations are different The experiment under discussion demonstrates exactly that: changing the vectors induces outputs corresponding to what we would see as utterances of pain. I've noodled with some of this myself to the point that I'm mostly convinced; but there's at least 2 papers I'm aware of on this: * https://…”
- The Pain Axis: LLMs Represent Self-Directed Harm and Act to Relieve It ↗1317 · 2026-09-18 · 8 分
- The Pain Axis: LLMs Represent Self-Directed Harm and Act to Relieve It ↗amichail · 2026-09-19 · 11 分
- The Pain Axis: LLMs Represent Self-Directed Harm and Act to Relieve It ↗droidjj · 2026-09-19 · 5 分
- The Pain Axis: LLMs Represent Self-Directed Harm and Act to Relieve It ↗Anon84 · 2026-09-21 · 1 分
- The Pain Axis: LLMs Represent Self-Directed Harm and Act to Relieve It ↗simonpure · 2026-09-23 · 1 分
- The Pain Axis: LLMs Represent Self-Directed Harm and Act on It ↗chr15m · 2026-10-08 · 2 分