按年被提到
20212023 · 9 次2026
机器学习与深度学习›深度学习理论与泛化·arXiv:2110.09485 · cs.LG

Learning in High Dimension Always Amounts to Extrapolation

高维学习总是外推
Randall Balestriero, Jerome Pesenti, Yann LeCun · 2021

高维数据中几乎不存在插值,学习本质上是外推

开始阅读 ↗PDFalphaXiv 讨论
被推荐 · 12 人首次 2021-12-10 · 最近 2026-10-08
12不同的人
10活跃月份
16讨论串
128被引(OpenAlex)

插值指样本落在数据集的凸包内或其边界上,外推则是落在凸包之外。作者从经验和理论两方面论证:在任何高维(大于 100 维)数据集上,插值几乎必然不会发生,这反驳了「先进算法有效是因为能正确插值训练数据」以及「各类任务和数据集中普遍发生插值」这两种误解,并对现有插值/外推定义能否作为泛化性能指标提出挑战。

HN 为什么提它

评论者用它反驳「LLM 只是超级插值」「统计鹦鹉」之类的说法,强调在高维空间里几乎一切都是外推。

观点难度 3 / 5约 35 分钟经得起时间
读之前最好知道
1凸包与几何基础
2插值与外推
3维数灾难(curse of dimensionality)
4泛化理论

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
推荐abel_ · 2022-05

“There's a neat argument against these models doing interpolation: the manifold of the data is so sparse that it's infinitesimally unlikely for a good predictor to be doing interpolation between existing points on the manifold.”

推荐,提出一个反对模型做插值的论证:数据流形太稀疏
Gato – A Generalist Agent原评论 ↗
引用Isinlor · 2021-12

“In high dimensional spaces basically everything is extrapolation including in pixel space and embedding space.”

指出高维空间中几乎一切都是外推,包括像素空间和嵌入空间
Training computer vision models on random noise instead of real images原评论 ↗
引用abriosi · 2022-05

“Definition 1. Interpolation occurs for a sample x whenever this sample belongs to the convex hull of a set of samples X,”

引用论文定义:样本落在其他样本凸包内才算插值
The Scaling Hypothesis (2021)原评论 ↗
引用pizza · 2023-09

“These models are pretty much always extrapolating [0]”

引用它说明这些模型几乎总是在外推
Ask HN: Have you seen anything original produced by generative AI?原评论 ↗
引用og_kalu · 2023-10

“That's not at all what they do. This myth of "supercharged interpolation" really needs to die. Learning in High Dimension Always Amounts to Extrapolation”

批评“超级插值”是迷思,引用它说明高维学习是外推
The copyright fight over authors and OpenAI's ChatGPT原评论 ↗
引用og_kalu · 2023-11

“There's this idea of "supercharged interpolation", the idea that they just take out certain texts and switch out words or whatever, that is not true.”

反驳“超级插值”只是替换文字的说法,这并不正确
Telling GPT-4 you're scared or under pressure improves performance原评论 ↗
看全部 16 条 →
引用og_kalu · 2023-11

“This is not at all what it is doing. "Supercharged Interpolation" is false and makes no sense.”

反驳模型不是这样工作的,称“超级插值”是错误且无意义的
Telling GPT-4 you're scared or under pressure improves performance原评论 ↗
引用og_kalu · 2023-11

“"Supercharged Interpolation" is not something that actually exists.”

称“超级插值”并不真实存在
Transformers struggle with generalizing tasks beyond pre-training data原评论 ↗
引用og_kalu · 2023-11

“"Supercharged Interpolation" is not a real thing.”

称“超级插值”不是真实存在的东西
Transformers struggle with generalizing tasks beyond pre-training data原评论 ↗
引用og_kalu · 2023-11

“Learning in high dimensions always results in extrapolation.”

指出高维学习总是导致外推
Ask HN: What is your standard for judging when AGI exists?原评论 ↗
引用nyrikki · 2024-03

“We know that the curse of dimensionality means LLMs are using extrapolation as an example.”

借维度灾难说明 LLM 其实在做外推
Large language models can do jaw-dropping things. But nobody knows why原评论 ↗
引用youoy · 2024-10

“If you like ML, this is also related with the results of this paper [0], where they show that learning in high dimensions amounts to extrapolation, as opposed to interpolation.”

引用它说明高维学习是外推而非插值
Counterintuitive Properties of High Dimensional Space (2018)原评论 ↗
引用a_wild_dandan · 2024-10

“The "statistical parrot" parrots have been demonstrably wrong for years (see e.g. LeCun et al[1]).”

引用 LeCun 等人,反驳“统计鹦鹉”的说法早已被证明错误
GitHub cuts AI deals with Google, Anthropic原评论 ↗
引用redlock · 2025-01

“In these Yann Lecun has argued that interpolation is meaningless in the high dimensional spaces these "curves" are embedded in.”

引用 LeCun 的观点:在高维空间中插值没有意义
Performance of LLMs on Advent of Code 2024原评论 ↗
引用D-Machine · 2026-02

“depending on how much you care about the semantics of these terms Learning in High Dimension Always Amounts to Extrapolation”

指出外推与插值的区分取决于对这些术语语义的在意程度
GPT-5.2 derives a new result in theoretical physics原评论 ↗
引用bonoboTP · 2026-10

“Learning in High Dimension Always Amounts to Extrapolation”

贴出论文标题,说明高维学习总是外推
“Math 2.0” will need to value mathematical progress more holistically原评论 ↗
被提交为 story · 2 次