“For example, see the paper Visualizing Loss Functions For Neural Networks as an example of the incredible insight to be gained with better visualization.”
Visualizing the Loss Landscape of Neural Nets
用滤波器归一化可视化神经网络损失曲面,并比较结构与训练参数的影响
论文提出一种简单的「filter normalization」方法,使不同网络的损失函数曲率可以被有意义地并排比较,并以多种可视化方式考察网络结构(如跳跃连接)和训练参数如何影响损失曲面的形状,以及损失曲面与泛化的关系。
评论者把它当作那些漂亮的损失曲面图的出处,用来说明可视化能带来洞见,也用来佐证跳跃连接(如 ResNet)能让损失曲面更平滑、更易优化。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“This is the one example I had in mind because of all the pretty pictures: Visualizing the Loss Landscape of Neural Nets”
“This paper has some nice visualizations.”
“There is some evidence that architectural tricks (like ResNet) that empirically help performance are making the loss landscape”
“TLDR - loss landscapes are nasty, but you can tame them with skip connections.”
“2) Visualizing the Loss Landscape of Neural Nets Visualizing the Loss Landscape of Neural Nets”
看全部 7 条 →
“People have been trying to visualize the loss and trajectory taken during optimization Visualizing the Loss Landscape of Neural Nets”
- Visualizing the Loss Landscape of Neural Nets [pdf] ↗stablemap · 2017-12-29 · 1 分