推荐gillesjacobs · 2023-06
“Hyperband [1] has been my go-to hyperparam optimization method over the past few years. Handily beats Bayesian search wherever I applied it”
称 Hyperband 是自己多年来的首选,实践中胜过贝叶斯搜索
Hyperband:用自适应资源分配与早停加速超参数随机搜索
论文不走贝叶斯优化路线,而是通过自适应资源分配和早停来加速随机搜索:把超参数优化表述为纯探索的非随机无限臂老虎机问题,把迭代次数、数据量或特征数等预设资源分配给随机采样的配置。作者提出 Hyperband 算法并分析其理论性质,在一系列深度学习和核方法问题上与流行的贝叶斯优化对比,观察到超过一个数量级的加速。
HN 评论者在聊超参数调优时反复推荐它,称其在实践中常胜过贝叶斯搜索,还有人提到它被用于谷歌的内部机器学习工具。
“Hyperband [1] has been my go-to hyperparam optimization method over the past few years. Handily beats Bayesian search wherever I applied it”
“hyperband - what google uses in their internal ML toolkits AFAIK”
“the one I'm talking about is called hyperband [0]”
“Hyperband - [2]- is an example.”