按年被提到
20132015 · 5 次2026
机器学习与深度学习›深度学习理论与泛化·arXiv:1312.6199 · cs.CV

Intriguing properties of neural networks

神经网络的有趣性质
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, Rob Fergus · 2013

发现不可察觉的微小扰动即可让神经网络误分类

开始阅读 ↗PDFalphaXiv 讨论
被推荐 · 22 人首次 2014-04-09 · 最近 2022-01-07
22不同的人
19活跃月份
22讨论串
8,137被引(OpenAlex)

论文报告了深度神经网络的两个反直觉性质。其一,按多种单元分析方法,高层的单个单元与单元的随机线性组合并无区别,说明语义信息存在于整个空间而非单个单元中;其二,网络学到的输入输出映射在相当程度上是不连续的:对图像施加通过最大化预测误差找到的、人眼不可察觉的扰动,就能让网络误分类,而且这类扰动并非学习的随机产物,同一个扰动还能让在不同数据子集上训练的另一个网络对同一输入误分类。

HN 为什么提它

HN 上的人把它当作对抗样本的原始论文,常拿它说明深度网络脆弱、扰动可跨网络迁移,并在讨论自动驾驶等场景的鲁棒性时引用。

实证研究难度 3 / 5约 40 分钟经得起时间
读之前最好知道
1深度神经网络与图像分类
2损失函数与梯度优化
3泛化与鲁棒性(robustness)的概念
4线性代数中的向量空间与范数

学习路径

读这篇之前,按顺序读书架上的 3 篇 · 参考文献来自 OpenAlex,顺序和要点由 Claude Code 排出

这条路径帮助理解单元分析的背景:无监督学习出的高层特征、卷积网络的可视化与 dropout,再读神经网络的两个反直觉性质。

  1. 1
    Building high-level features using large scale unsupervised learning用大规模无监督学习构建高层特征 · 2011

    论文的单元分析沿用了寻找使单个神经元激活最大的输入这类做法。

    读抓如何通过最大激活输入分析单个神经元。

  2. 2
    Visualizing and Understanding Convolutional Networks卷积网络的可视化与理解 · 2013

    提供卷积网络内部单元可视化的另一种方法,与论文的单元分析相关。

    读抓反卷积式可视化,理解单个单元对应的图像模式。

  3. 3
    Improving neural networks by preventing co-adaptation of feature detectors通过防止特征检测器的共适应来改进神经网络 · 2012

    dropout 是论文研究的网络训练方法之一,需要知道其作用。

    读抓随机丢弃单元的做法;实验部分略读。

  4. 4
    Intriguing properties of neural networks神经网络的有趣性质 · 2013 · 就是这篇

    发现不可察觉的微小扰动即可让神经网络误分类,且单元与单元的随机线性组合没有区别。

    读抓第一个性质(空间而非单元携带信息)和第二个性质(对抗样本)的实验。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
推荐sotte · 2014-09

“If you're interested in the generalization ability of neural networks I can recommend the following paper:”

向关心神经网络泛化能力的人推荐这篇论文
A visual proof that neural nets can compute any function原评论 ↗
推荐abeppu · 2014-11

“Naturally the paper is where the good stuff is.”

称好东西都在论文里,不在博客摘要里
The Flaw Lurking in Every Deep Neural Net原评论 ↗
推荐AndrewOMartin · 2014-12

“Don't forget "Intriguing properties of neural networks", otherwise known as "Does Deep Learning have deep flaws?".”

提醒别忘了这篇,也就是深度学习有没有深层缺陷的那篇
Deep Learning Reading List原评论 ↗
推荐LoSboccacc · 2019-12

“start here: Intriguing properties of neural networks”

推荐从这篇开始读,它是对抗样本的源头
Autonomous DeLorean drives sideways to move forward原评论 ↗
引用ot · 2014-04

“a recent paper [1] shows that if we feed to an object classifying NN an image with crafted but unnoticeable perturbations the predictions change completely.”

引用它说明难以察觉的微小扰动能让分类预测完全改变
Neural Networks, Manifolds, and Topology原评论 ↗
引用colah3 · 2015-01

“This paper shows that one can craft noise which will fool a neural network. But it isn't noise that is found in real life.”

指出它造出的噪声能骗过网络,但现实中并不存在这种噪声
A.I. Has Grown Up and Left Home原评论 ↗
看全部 20 条 →
引用yummyfajitas · 2015-03

“Witness, for example, all the recent "we can fool deep learning image recognition systems" papers.”

拿这类骗过图像识别的论文说明算法的错误和人类偏差不同
Life Lessons from Machine Learning原评论 ↗
引用Houshalter · 2015-03

“To be clear I meant this paper as the original paper for adversarial images.”

澄清自己指的是这篇,它是对抗图像的原始论文
Images that fool computer vision raise security concerns原评论 ↗
引用dllu · 2016-03

“Even if there is no overfitting, deep neural networks seem to learn a surprisingly discontinuous function.”

引用它说明即使没有过拟合,深度网络学到的函数也出奇地不连续
Lee Sedol Beats AlphaGo in Game 4原评论 ↗
引用jkldotio · 2016-05

“has examples where you get radically different classifications that I don't think would occur naturally or survive a blur with some random element”

指出其中分类结果天差地别的例子,自然界不会出现,也经不起模糊
End to End Learning for Self-Driving Cars [pdf]原评论 ↗
引用obastani · 2016-10

“AI/ML algos cannot magically guarantee safety, for example, deep nets are known to be very un-robust: Intriguing properties of neural networks.”

引用它说明深度网络很不稳健,AI/ML 不能保证安全
All Tesla Cars Being Produced Now Have Full Self-Driving Hardware原评论 ↗
引用obastani · 2016-10

“For example, self-driving cars based on machine learning may be susceptible to adversarial attacks that cause the car to behave unpredictably [1,2].”

引用它说明自动驾驶等 ML 系统可能受对抗攻击而行为失常
All Tesla Cars Being Produced Now Have Full Self-Driving Hardware原评论 ↗
引用BickNowstrom · 2016-11

“From one of the canonical papers on adversarial images:”

称它是对抗图像的经典论文之一,扰动可跨网络迁移
Real and Stealthy Attacks on State-Of-the-Art Face Recognition [pdf]原评论 ↗
引用mholt · 2017-04

“See Intriguing properties of neural networks and [OTHER_PAPER]”

给出对抗样本可迁移性的参考,这限制了对抗训练
Optical illusions that flummox computers原评论 ↗
引用yorwba · 2017-10

“We can cause the network to misclassify an image by applying a certain imperceptible perturbation, which is found by maximizing the network's prediction error.”

引用它说明最大化预测误差得到的难以察觉扰动会导致误分类
AI Model Fundamentally Cracks Captchas, Scientists Say原评论 ↗
引用haraldurt · 2017-11

“Adversarial examples have been shown to, for instance, be transferable across different networks with different hyperparameters (e.g., number of layers) trained on disjoint subsets of a training set [0, section 4.2].”

引用第 4.2 节:对抗样本可在不同超参数、不同数据子集的网络间迁移
How Adversarial Attacks Work原评论 ↗
引用pinouchon · 2017-11

“Szegedy et al [9] has the following abstract:”

贴出 Szegedy 等人的摘要,说明语义信息在单元空间而非单个神经元里
Feature Visualization: How neural nets build up their understanding of images原评论 ↗
引用bagnalla · 2017-12

“is a major research topic at the moment.”

让读者了解对抗样本,称其是当前的重要研究课题
Interview with Xavier Leroy原评论 ↗
引用sennight · 2021-03

“This bus is an ostrich:”

用公交车被认成鸵鸟的例子说明微小扰动能让网络误分类
Super Resolution原评论 ↗
引用YeGoblynQueenne · 2022-01

“adversarial examples [2],”

引用对抗样本,说明深度网络有隐藏的脆弱点
Deep learning can’t be trusted, brain modelling pioneer says原评论 ↗