按年被提到
20202020 · 1 次2026
机器学习与深度学习›深度学习理论与泛化·arXiv:2002.08791 · cs.LG

Bayesian Deep Learning and a Probabilistic Perspective of Generalization

Andrew Gordon Wilson, Pavel Izmailov · 2020
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2020-02-21 · 最近 2021-09-21
2不同的人
2活跃月份
2讨论串
380被引(OpenAlex)
arXiv 摘要(候选层不写导读)

The key distinguishing property of a Bayesian approach is marginalization, rather than using a single setting of weights. Bayesian marginalization can particularly improve the accuracy and calibration of modern deep neural networks, which are typically underspecified by the data, and can represent many compelling but different solutions. We show that deep ensembles provide an effective mechanism for approximate Bayesian marginalization, and propose a related approach that further improves the predictive distribution by marginalizing within basins of attraction, without significant overhead. We also investigate the prior over functions implied by a vague distribution over neural network weights, explaining the generalization properties of such models from a probabilistic perspective. From this perspective, we explain results that have been presented as mysterious and distinct to neural network generalization, such as the ability to fit images with random labels, and show that these results can be reproduced with Gaussian processes. We also show that Bayesian model averaging alleviates double descent, resulting in monotonic performance improvements with increased flexibility. Finally, we provide a Bayesian perspective on tempering for calibrating predictive distributions.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论yldedly · 2021-09

“DD disappears when doing Bayesian model averaging: https://arxiv.org/pdf/2002.08791.pdf It seems DD is a phenomenon specific to point estimates. >overfitting problem is really in terms of model flexibility versus data complexity, and not necessarily in terms of number of parameters per se versus sample size Yep, well put.”

An overview of the theory of overparameterized machine learning原评论 ↗
被提交为 story · 1 次