按年被提到
20192020 · 1 次2026
机器学习与深度学习›深度学习理论与泛化·arXiv:1903.08560 · math.ST

Surprises in High-Dimensional Ridgeless Least Squares Interpolation

Trevor Hastie, Andrea Montanari, Saharon Rosset, Ryan J. Tibshirani · 2019
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2020-07-26 · 最近 2026-04-26
2不同的人
2活跃月份
2讨论串
556被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Interpolators -- estimators that achieve zero training error -- have attracted growing attention in machine learning, mainly because state-of-the art neural networks appear to be models of this type. In this paper, we study minimum $\ell_2$ norm ("ridgeless") interpolation in high-dimensional least squares regression. We consider two different models for the feature distribution: a linear model, where the feature vectors $x_i \in {\mathbb R}^p$ are obtained by applying a linear transform to a vector of i.i.d. entries, $x_i = Σ^{1/2} z_i$ (with $z_i \in {\mathbb R}^p$); and a nonlinear model, where the feature vectors are obtained by passing the input through a random one-layer neural network, $x_i = \varphi(W z_i)$ (with $z_i \in {\mathbb R}^d$, $W \in {\mathbb R}^{p \times d}$ a matrix of i.i.d. entries, and $\varphi$ an activation function acting componentwise on $W z_i$). We recover -- in a precise quantitative way -- several phenomena that have been observed in large-scale neural networks and kernel machines, including the "double descent" behavior of the prediction risk, and the potential benefits of overparametrization.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论cs702 · 2020-07

“>... all DNNs face the issue of overfitting as they learn, which is when performance on one data set increases but the network's performance fails to generalize (often measured by the divergence of performance on training vs testing data sets). This ubiquitous problem in DNNs is often solved by experimenters via "noise injections" in the form of noisy or…”

The Overfitted Brain: Dreams evolved to assist generalization原评论 ↗
评论hodgehog11 · 2026-04

“Hastie was actually lead author of an excellent paper that discusses the underlying phenomenon in the context of least-squares linear regression: https://arxiv.org/abs/1903.08560 It really isn't so mysterious once you begin to examine how the rule of thumb for the bias-variance tradeoff (remember that it is the relationship with model size that is curiou…”

There Will Be a Scientific Theory of Deep Learning原评论 ↗