“is also a great insight into this phenomena”
Measuring the tendency of CNNs to Learn Surface Statistical Regularities
用傅里叶滤波证明 CNN 倾向于抓住表层统计规律
深度 CNN 泛化得很好,却对对抗扰动极其敏感,作者想弄清它们在没有学到高层语义的情况下为何能泛化。他们用傅里叶滤波在 SVHN 和 CIFAR-10 上构造高层语义相同、表层统计规律不同的数据集变体(低频版和随机滤波版),发现 CNN 倾向于抓住训练集的傅里叶图像统计,泛化差距最高达 28%;显著加深网络对缩小这一差距作用很小。
评论者引用它说明 CNN 主要学到的是表层统计而非语义特征,进而质疑深层是否真的捕捉了语义,并联系数据增强与对抗样本。
学习路径
读这篇之前,按顺序读书架上的 4 篇 · 参考文献来自 OpenAlex,顺序和要点由 Claude Code 排出先看对抗样本的发现和快速生成方法,再看被愚弄的网络和通用对抗扰动,才能理解目标论文为什么用傅里叶滤波来衡量 CNN 对表层统计规律的依赖。
- 1
- 2
- 3
对人眼不可辨认的图像给出高置信度预测,说明网络依赖的并非人类意义上的语义。
读进化算法生成的噪声图及其高置信度分类的实验。
- 4
- 5Measuring the tendency of CNNs to Learn Surface Statistical Regularities
目标论文:用傅里叶滤波改变图像的表层统计规律,衡量 CNN 对它的依赖。
读傅里叶滤波后的训练和测试设置、泛化差距,以及它对 CNN 倾向抓表层规律的结论。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“we don't even know why they generalise in the first place! See: [OTHER_PAPER], Measuring the tendency of CNNs to Learn Surface Statistical Regularities”
“they confirmed that even networks that generalize well are learning mostly surface-level statistics.”
“This problem is discussed in [0], which creates a lot of problems for people who want to believe that some deep, internal layer of a neural net can capture semantically relevant features.”
“This paper gives an example of exactly this issue for CNN models for image processing [0].”
“I'm thinking in the spirit of: < Measuring the tendency of CNNs to Learn Surface Statistical Regularities >.”
看全部 9 条 →
“see also Measuring the tendency of CNNs to Learn Surface Statistical Regularities”
“In light of [0], there is both theoretical and experimental reason to believe that CNNs are mostly picking up fourier decomposition.”
“Some links: Measuring the tendency of CNNs to Learn Surface Statistical Regularities [OTHER_PAPER]”
- CNNs learn surface statistical regularities, not high-level abstract concepts ↗nafizh · 2018-01-16 · 2 分
- Measuring the Tendency of CNNs to Learn Surface Statistical Regularities ↗tim_sw · 2018-01-17 · 66 分