按年被提到
20162016 · 6 次2026
机器学习与深度学习›深度学习理论与泛化·arXiv:1608.08225 · cond-mat.dis-nn

Why does deep and cheap learning work so well?

为什么深度而廉价的学习效果这么好?
Henry W. Lin, Max Tegmark, David Rolnick · 2016

用物理视角解释深度学习为何有效

开始阅读 ↗PDFalphaXiv 讨论
被推荐 · 12 人首次 2016-08-31 · 最近 2023-04-26
12不同的人
14活跃月份
15讨论串
408被引(OpenAlex)

论证深度学习的成功不仅取决于数学,也取决于物理:实际关心的函数类往往只需指数级更少的参数就能近似,即所谓廉价学习。对称性、局部性、组合性等物理中常见的性质会转化为极简单的神经网络;当数据由某种层级式过程生成时,深度网络比浅层网络更高效。作者用信息论形式化这些论点,讨论其与重整化群的关系,并证明了若干不可展平定理。

HN 为什么提它

评论者把它当作解释深度学习为何有效的最有说服力的物理视角之一:自然数据的分布受物理定律约束,因而容易学习。

观点难度 4 / 5约 90 分钟经得起时间
读之前最好知道
1神经网络通用逼近定理
2信息论基础(熵、互信息)
3统计物理与重整化群(renormalization group)
4概率图模型与层级生成过程

学习路径

读这篇之前,按顺序读书架上的 1 篇 · 参考文献来自 OpenAlex,顺序和要点由 Claude Code 排出

候选里只有一篇用神经网络求解量子多体问题的论文与目标论文有较明确的联系,所以路径很短。

  1. 1

    用神经网络表示量子多体波函数,是物理与神经网络结合的实例,与目标论文的物理视角呼应。

    读抓用受限玻尔兹曼机表示波函数的想法;训练细节略读。

  2. 2
    Why does deep and cheap learning work so well?为什么深度而廉价的学习效果这么好? · 2016 · 就是这篇

    用物理视角解释深度学习为何有效。

    读抓对称性、局部性、组合性如何使网络只需指数级更少的参数,以及与重整化的类比。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
强烈推荐cs702 · 2023-04

“The most persuasive argument I've seen for this view -- our view -- is laid out in this paper by Lin, Tegmark, and Rolnick”

称这是支持该观点最有说服力的论证
A Cookbook of Self-Supervised Learning原评论 ↗
推荐cs702 · 2017-11

“you might also enjoy really interesting paper coauthored by the same person”

推荐同一作者合著的另一篇有趣论文
Shut up and calculate: an extreme mathematical universe原评论 ↗
推荐mikhailfranco · 2020-10

“Another interesting explanation for deep learning's success in the physical world:”

推荐它作为深度学习在物理世界成功的另一种解释
Why deep learning works even though it shouldn’t原评论 ↗
引用sgt101 · 2016-09

“there was a paper by Max Tegmark on HN yesterday with some new ideas and results, but I haven't had time to read it yet.”

提到 Tegmark 昨天在 HN 上有篇论文带来新想法,自己还没读
Is artificial intelligence permanently inscrutable?原评论 ↗
引用throwaway000002 · 2016-10

“He comments on the video linking to two papers on arXiv which relate to the material in the lecture: Why does deep and cheap learning work so well? and [OTHER_PAPER]”

指出视频下评论链接了两篇与讲座内容相关的 arXiv 论文
[MIT Lecture] Connections between physics and deep learning原评论 ↗
引用cs702 · 2016-10

“That does not appear to be true in theory (due to the laws of physics, e.g., see Why does deep and cheap learning work so well? ) nor in practice”

引用它说明受物理定律影响,该说法在理论和实践中都不成立
The Neural Network Zoo原评论 ↗
看全部 12 条 →
引用BickNowstrom · 2016-12

“3. Why does deep and cheap learning work so well? Physics and Deep Learning are well entwined.”

引用它说明物理与深度学习紧密交织
OpenAI Universe原评论 ↗
引用epberry · 2016-12

“there are also many researchers poking and prodding at deep learning theory [1, 2].”

把它列为研究者探究深度学习理论的工作之一
OpenAI Universe原评论 ↗
引用colorincorrect · 2017-04

“Perhaps this paper provides an explanation?”

猜测这篇论文或许能给出解释
Unreasonable Ineffectiveness of Machine Learning in Computer Systems Research原评论 ↗
引用cs702 · 2017-05

“this concentration of natural data distributions might be a result of the laws of Physics of the universe in which we happen to live: Why does deep and cheap learning work so well?”

提出自然数据分布的集中可能源于我们所处宇宙的物理定律
Understanding deep learning requires re-thinking generalization原评论 ↗
引用mikhailfranco · 2017-08

“Tegmark & Lin's discussion of how deep learning maps onto the physical world: Why does deep and cheap learning work so well? [1][2]”

引用 Tegmark 和 Lin 关于深度学习如何映射到物理世界的讨论
Deep Learning and Quantum Entanglement [pdf]原评论 ↗
引用laclar · 2018-10

“but perhaps it could be of interest to you. Why does deep and cheap learning work so well?”

说自己没读过,但觉得对方可能感兴趣
How I Transitioned from Physics Academia to the ML Industry原评论 ↗
被提交为 story · 4 次