引用EMIRELADERO · 2025-04
“the single consistent finding across different models is that they generalize beyond the training data for at least those simple arithmetic operations.”
指出不同模型一致表明,至少简单算术上它们能泛化到训练数据之外
数字在LLM里以螺旋线表示,用“时钟”算法做加法
作者逆向分析三个中等规模大模型如何做加法。首先发现数字在模型里被表示成一种广义螺旋线(generalized helix),它对加减法有强因果影响,对整数除法、乘法和模运算也有因果相关性。随后提出模型通过“Clock”算法操作螺旋线来计算 a+b:把 a、b 的螺旋线变换成 a+b 的答案螺旋线,再读出为 logits;他们用这些螺旋线建模关键的 MLP 输出、注意力头输出乃至单个神经元预激活,并用因果干预验证。这是对大模型数学能力的首个表征层面的解释。
HN 评论者用它说明大模型做加法时用的是三角函数式的内部特征,而不是记住答案,并泛化到训练数据之外。
“the single consistent finding across different models is that they generalize beyond the training data for at least those simple arithmetic operations.”
“There have been multiple studies on how LLMs add numbers.”
“functional analyses which have identified how LLMs do arithmetic - Language Models Use Trigonometry to Do Addition - and how refusal happens: [OTHER_PAPER]”
“Models do have some emergent capabilities[1]”