按年被提到
20192024 · 10 次2026
AI 安全、伦理与社会影响›通用智能与意识·arXiv:1911.01547 · cs.AI

On the Measure of Intelligence

论智能的度量
François Chollet · 2019

把智能定义为技能获取效率,并提出 ARC 基准来衡量

开始阅读 ↗PDFalphaXiv 讨论
被推荐 · 35 人首次 2019-11-06 · 最近 2026-09-16
35不同的人
26活跃月份
43讨论串
544被引(OpenAlex)

Chollet 回顾心理学和 AI 中定义与衡量智能的方法,指出只看某项任务上的技能会被先验知识和海量训练数据掩盖系统自身的泛化能力。他基于算法信息论提出把智能看作技能获取效率的形式化定义,给出通用 AI 基准应遵循的准则,并推出 Abstraction and Reasoning Corpus(ARC)基准。

HN 为什么提它

HN 上的人在讨论 AI 是否真的有智能、基准测试是否有意义时反复引用它,称它是对这一问题写得最好的文章,并常把它和 ARC 基准一起提。

观点难度 3 / 5约 150 分钟经得起时间
读之前最好知道
1机器学习与基准测试(benchmark)基础
2算法信息论(Algorithmic Information Theory)基本概念
3心理测量学中智能测试的基本概念
4泛化(generalization)与先验知识的概念

学习路径

读这篇之前,按顺序读书架上的 3 篇 · 参考文献来自 OpenAlex,顺序和要点由 Claude Code 排出

Chollet 批评深度学习只在局部泛化、容易被表层统计规律欺骗,这条路径先补上「脆弱性」和「类人智能」两方面的背景,再读他对智能的重新定义。

  1. 1
    Intriguing properties of neural networks神经网络的有趣性质 · 2013

    对抗样本说明网络的泛化很脆弱,是 Chollet 质疑只看任务表现的背景之一。

    读读对抗样本部分:微小扰动如何让分类器出错;单元分析部分可略读。

  2. 2
    Measuring the tendency of CNNs to Learn Surface Statistical Regularities衡量 CNN 学习表层统计规律的倾向 · 2017

    显示 CNN 倾向抓表层统计规律而非高层抽象,对应目标论文区分「技能」与「泛化能力」的动机。

    读抓傅里叶滤波实验的思路与结论:网络学到的是什么、没学到什么。

  3. 3
    Building Machines That Learn and Think Like People构建像人一样学习和思考的机器 · 2016

    认知科学视角的类人 AI 主张(核心知识、因果模型、组合性),Chollet 的先验与核心知识概念与此一脉相承。

    读读关于核心能力(直觉物理、直觉心理)和「学得快」的论述;各子领域细节可跳过。

  4. 4
    On the Measure of Intelligence论智能的度量 · 2019 · 就是这篇

    把智能定义为技能获取效率,并提出 ARC 基准。

    读抓智能定义的形式化(先验、经验、泛化难度)与 ARC 的设计思路。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
强烈推荐nl · 2021-04

“I think Chollet's "On the Measure of Intelligence" paper[1] is the best writing I've seen on this”

强烈推荐,称这是自己见过关于这个问题最好的文章
Why AI is harder than we think原评论 ↗
强烈推荐TacticalCoder · 2024-03

“a wonderful paper (which I probably found here on HN) on the subject”

称赞这是一篇精彩的论文,Chollet 的测试人类易解而 AI 难解
AIs ranked by IQ; AI passes 100 IQ for first time, with release of Claude-3原评论 ↗
强烈推荐mrandish · 2025-01

“is the best overview of the challenges I've read. Highly recommended.”

强烈推荐,称这是读过的对相关挑战最好的概述
Results of "Humanity's Last Exam" benchmark published原评论 ↗
推荐glial · 2019-11

“Francois Chollet has a good recent paper about this [1].”

推荐 Chollet 近期这篇关于该话题的好论文
What Would the Father of Cybernetics Think About A.I. Today?原评论 ↗
推荐Isinlor · 2019-11

“I agree with him. We should move away from benchmarks that involve training and evaluating algorithms on the same datasets.”

赞同 Chollet,主张摆脱在同一数据集上训练评估的基准
AI is mostly about curve fitting (2018)原评论 ↗
推荐visarga · 2020-06

“There is a nice paper on this topic.”

推荐这篇关于该话题的好论文
Human-Level Intelligence or Animal-Like Abilities? (2018)原评论 ↗
看全部 20 条 →
推荐m3at · 2020-12

“This is a great meta paper questioning the goal of the field itself, and proposing ways to formally evaluate intelligence in a computational sense.”

推荐它:质疑领域目标本身,并提出形式化评估智能的方法
Ask HN: What's the best paper you've read in 2020?原评论 ↗
推荐visarga · 2022-09

“I recommend "On the Measure of Intelligence" by François Chollet”

推荐 Chollet 这篇论文,核心是智能即技能习得效率
The Problem with Intelligence原评论 ↗
推荐sanxiyn · 2023-04

“You know, François Chollet literally did this, and people don't listen. People should listen to Chollet more.”

指出 Chollet 早已做过此事,主张多听他的
Eight Things to Know About Large Language Models [pdf]原评论 ↗
推荐TacticalCoder · 2024-01

“Start at page 48 from this paper to see lots of cool tests, totally trivial for us humans to solve, that apparently AI have issues with.”

推荐从第 48 页看起,那里有人类轻松而 AI 吃力的测试
The IQ Test That AI Can't Pass原评论 ↗
推荐levocardia · 2024-03

“I found it very useful for thinking about what it means for an AI to be generally intelligent.”

称它对思考通用智能意味着什么很有用
How far are we from intelligent visual deductive reasoning?原评论 ↗
推荐daveguy · 2026-03

“You should go back and read the original ARC-AGI paper to see what this is about+.”

建议回头读 ARC-AGI 原论文以了解其目的
ARC-AGI-3原评论 ↗
推荐gobdovan · 2026-05

“The Cholet paper is a good attempt”

认为这是一次不错的尝试:智能应衡量技能获取而非技能本身
To have a moral stance on AI is to be an outcast, and it sucks原评论 ↗
推荐surprisetalk · 2026-09

“I rather like this paper, but I think it is generous to say their benchmarks measure "intelligence".”

喜欢这篇论文,但认为说其基准测的是“智能”有些抬举
Intelligence per Watt: Measuring Intelligence Efficiency of Local AI原评论 ↗
批评whiplash451 · 2025-07

“ARC-AGI is a laudable effort, but its fundamental premise is indeed debatable:”

批评 ARC-AGI 的基本前提值得商榷,尽管努力可嘉
François Chollet: The Arc Prize and How We Get to AGI [video]原评论 ↗
批评cubefox · 2025-07

“Which makes these benchmarks mostly post hoc rationalizations.”

François Chollet: The Arc Prize and How We Get to AGI [video]原评论 ↗
引用andra1 · 2020-06

“Francois Chollet tries to address some of these problems by tackling how intelligence is measured.”

拿它回应智能度量的问题:智能是技能习得效率而非任务表现
Human-Level Intelligence or Animal-Like Abilities? (2018)原评论 ↗
引用YeGoblynQueenne · 2020-11

“it (also from Chollet) [2] makes a strong claim about the necessity of”

引用 Chollet 的强主张:智能需要知识先验
AI Expert Roadmap原评论 ↗
引用yamrzou · 2021-06

“This reminds me of The Abstraction and Reasoning Corpus [1] by François Chollet, accompanying his paper On the Measure of Intelligence [2]”

想起 Chollet 的智能度量框架及其配套的 ARC 语料库
A first lesson in meta-rationality原评论 ↗
引用sinenomine · 2023-01

“a recent attempt at designing a psychometric test encoding human cognitive priors: On the Measure of Intelligence - the ARC dataset.”

把 ARC 数据集当作编码人类认知先验的心理测量尝试
What Is AGI-Hard原评论 ↗