“This paper puts words to something I’ve noticed repeatedly with LLMs, particularly Qwen3.6. When I read its reasoning, it appears to recognise the mistake and then carry on as if it hadn’t noticed it at all. > models often determine their answers based on implicit biases tied to question templates, then construct reasoning chains to justify their predete…”
Chain-of-Thought Reasoning In The Wild Is Not Always Faithful
Recent studies indicate that when faced with explicit biases in prompts, models often omit mentioning these biases in their Chain-of-Thought (CoT) output, revealing that verbalized reasoning can give an incorrect picture of how models arrive at conclusions (unfaithfulness). In this work, we show that unfaithful CoT also occurs on naturally worded, non-adversarial prompts without adding artificial biases or editing model outputs. We find that when separately presented with the questions "Is X bigger than Y?" and "Is Y bigger than X?", models sometimes produce superficially coherent arguments to justify systematically answering Yes to both or No to both, despite the contradiction. We present preliminary evidence that this is due to models' implicit biases towards Yes or No, labeling this Implicit Post-Hoc Rationalization. Our results reveal rates up to 13% for production models, and while frontier models are more faithful, none are entirely so, including thinking models like DeepSeek R1 (0.37%) and Sonnet 3.7 with thinking (0.04%). We also investigate Unfaithful Illogical Shortcuts, where models use subtly illogical reasoning to make speculative answers to hard math problems seem rigorously proven. Our findings indicate that while CoT can be useful for assessing outputs, it is not a complete account of the internal process that produced the model's answer and should be used with caution in agentic or safety-critical settings.
候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。
HN 书评
没有评论直接链到它;下面是它被提交的讨论串里回复最多的评论(不计入统计)“I thought this was already widely known? From March last year: https://transformer-circuits.pub/2025/attribution-graphs/biology.html#dives-cot There's no reason to believe the model's self-reported "thinking" bears any relation to the mechanics by which it arrived at some output.”
“Of course not. Because the article uses the words 'thought' and 'reasoning' and even 'faithful' to mean something other than their normal meanings, but then expects them to behave exactly the same. Every field has terms of art, and 'reasoning' is one for LLMs. But that doesn't mean it has the same properties as 'reasoning' in other contexts, because you'…”
- Chain-of-Thought Reasoning in the Wild Is Not Always Faithful ↗flyingpumba · 2025-03-14 · 2 分
- Chain-of-Thought Reasoning in the Wild Is Not Always Faithful ↗florianherrengt · 2026-08-19 · 12 分