按年被提到
20182018 · 2 次2026
机器学习与深度学习›深度学习理论与泛化·arXiv:1811.03804 · cs.LG

Gradient Descent Finds Global Minima of Deep Neural Networks

Simon S. Du, Jason D. Lee, Haochuan Li, Liwei Wang, Xiyu Zhai · 2018
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2018-11-12 · 最近 2018-12-08
2不同的人
2活跃月份
2讨论串
284被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Gradient descent finds a global minimum in training deep neural networks despite the objective function being non-convex. The current paper proves gradient descent achieves zero training loss in polynomial time for a deep over-parameterized neural network with residual connections (ResNet). Our analysis relies on the particular structure of the Gram matrix induced by the neural network architecture. This structure allows us to show the Gram matrix is stable throughout the training process and this stability implies the global optimality of the gradient descent algorithm. We further extend our analysis to deep residual convolutional neural networks and obtain a similar convergence result.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论mlthoughts2018 · 2018-12

“I suspect actually there is some connection between optimal batch size and the proofs of this paper regarding how deep NN will provably converge to zero training loss with gradient descent (not stochastic gradient descent). https://arxiv.org/abs/1811.03804”

A Bayesian Perspective on Generalization and Stochastic Gradient Descent原评论 ↗
被提交为 story · 1 次