引用yorwba · 2025-06
“The ARC-AGI-2 paper ARC-AGI-2 uses a non-representative sample, success rate differs widely across participants”
指出 ARC-AGI-2 论文用的是非代表性样本,不同参与者成功率差异很大
ARC-AGI 基准的升级版,人类易解而当前 AI 很难
论文介绍 ARC-AGI-2,它保留前代的输入输出对任务格式,但新增并扩充了一批任务,用来更细粒度地评估更高层次的抽象推理能力。作者给出大规模人类测试结果作为基线,说明这些任务对人类可做、对当前 AI 系统却很困难。
评论者引用它讨论人类解题率的样本是否有代表性,也用它说明上一代模型在 v2 上得分低于 5%。
“The ARC-AGI-2 paper ARC-AGI-2 uses a non-representative sample, success rate differs widely across participants”
“ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systems”
“As far as ARC-AGI goes it looks like last years models were scoring <5% against their v2 benchmark: ARC-AGI-2”