引用WarmWash · 2026-05
“Opus is estimated to be around 4T parameters, and 5.5 around 9T.”
引用探针估计:Opus 约 4T 参数,5.5 约 9T
用难压缩的冷门事实考模型,粗略估计闭源模型参数量
论文认为存储 F 条事实至少需要 F/(每参数比特数)个权重,因此事实记忆能力给出参数量的下界。作者构建了含 1,400 道、分 7 个冷门程度档次的事实题基准 IKP,在 93 个开源模型(135M 到 1,600B)上校准准确率到参数量的对数线性映射,R^2 为 0.910。该方法刻意设计得较粗:90% 预测区间约为上下 3 倍,只能恢复数量级和相对排名;对 MoE 模型,总参数量比激活参数量更能预测知识量,并给出主要闭源前沿模型的有效知识容量区间。
评论者用它来估算闭源模型的参数量:用难以压缩的冷门事实考模型,再用开源模型校准。
“Opus is estimated to be around 4T parameters, and 5.5 around 9T.”
“Oh ya, this is like the approach from the Incompressible Knowledge Probes [0] paper - smart!”
“The short of it is by using hard facts knowledge that is difficult to compress, and then quizzing models on these facts and calibrating against a bunch of open models, you can kind of feel out the size of closed models.”