按年被提到
20172024 · 3 次2026
机器学习与深度学习›深度学习理论与泛化·arXiv:1712.09913 · cs.LG

Visualizing the Loss Landscape of Neural Nets

神经网络损失曲面的可视化
Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, Tom Goldstein · 2017

用滤波器归一化可视化神经网络损失曲面,并比较结构与训练参数的影响

开始阅读 ↗PDFalphaXiv 讨论
被推荐 · 8 人首次 2017-12-29 · 最近 2025-11-02
8不同的人
8活跃月份
8讨论串
1,216被引(OpenAlex)

论文提出一种简单的「filter normalization」方法,使不同网络的损失函数曲率可以被有意义地并排比较,并以多种可视化方式考察网络结构(如跳跃连接)和训练参数如何影响损失曲面的形状,以及损失曲面与泛化的关系。

HN 为什么提它

评论者把它当作那些漂亮的损失曲面图的出处,用来说明可视化能带来洞见,也用来佐证跳跃连接(如 ResNet)能让损失曲面更平滑、更易优化。

实证研究难度 3 / 5约 40 分钟经得起时间
读之前最好知道
1神经网络训练基础
2损失函数与优化(SGD)
3跳跃连接与 ResNet
4泛化与批大小、学习率的关系
5线性代数基础(向量范数、随机方向)

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
推荐hadsed · 2018-02

“For example, see the paper Visualizing Loss Functions For Neural Networks as an example of the incredible insight to be gained with better visualization.”

拿它说明更好的可视化能带来深刻的洞察
Greedy, Brittle, Opaque, and Shallow: The Downsides to Deep Learning原评论 ↗
推荐nerdponx · 2024-02

“This is the one example I had in mind because of all the pretty pictures: Visualizing the Loss Landscape of Neural Nets”

推荐它作为例子,因为里面有很多漂亮的图
Training LLMs to generate text with citations via fine-grained rewards原评论 ↗
推荐Imnimo · 2024-05

“This paper has some nice visualizations.”

推荐这篇论文,里面的可视化做得不错
Gradient descent visualization原评论 ↗
引用currymj · 2019-01

“There is some evidence that architectural tricks (like ResNet) that empirically help performance are making the loss landscape”

引用它说明 ResNet 这类结构技巧可能让损失曲面更平滑
An Idiot’s guide to Support vector machines (2003) [pdf]原评论 ↗
引用muppet_frog · 2020-10

“TLDR - loss landscapes are nasty, but you can tame them with skip connections.”

概括:损失曲面很糟,但 skip connection 能驯服它
Why deep learning works even though it shouldn’t原评论 ↗
引用dontwearitout · 2024-06

“2) Visualizing the Loss Landscape of Neural Nets Visualizing the Loss Landscape of Neural Nets”

列出它,作为与相似解的盆地相关的损失曲面可视化
Grokfast: Accelerated Grokking by Amplifying Slow Gradients原评论 ↗
看全部 7 条 →
引用blackbear_ · 2025-11

“People have been trying to visualize the loss and trajectory taken during optimization Visualizing the Loss Landscape of Neural Nets”

指出已有人尝试可视化损失和优化过程中的轨迹,并引用它
Backpropagation is a leaky abstraction (2016)原评论 ↗
被提交为 story · 1 次