引用kastnerkyle · 2014-08
“has tried something similar to [Do Deep Networks Need to Be Deep?] for ImageNet scale networks.”
提到有人对 ImageNet 规模的网络做过类似尝试
浅层前馈网络可以学会深层网络的复杂函数,达到相近精度
这篇扩展摘要表明,浅层前馈网络能学会此前由深度网络学到的复杂函数,并达到此前只有深度模型才能达到的精度,有些情况下总参数量也与原深度模型相近。作者在 TIMIT 音素识别任务上训练出浅层全连接网络,表现与精心设计的深度卷积结构相近,并据此推测很可能存在比现有方法更好的浅层网络训练算法。
评论者用它说明浅层网络可以模仿深层网络(类似知识压缩),也有人据此认为深层网络只是更容易训练出最优解。
“has tried something similar to [Do Deep Networks Need to Be Deep?] for ImageNet scale networks.”
“From that paper, it seems like it's just that deep nets are easier to train to find state of the art solutions than shallow nets.”
“it's better to link to the abstract rather than directly to the PDF.”
“a recent paper showed that a simpler shallow net can perform as well as a deep net if a deep net is trained first[1].”