“A couple recent research papers on the topic are pretty readable:”
Practical Bayesian Optimization of Machine Learning Algorithms
用高斯过程贝叶斯优化自动调超参,可达到甚至超过专家水平
论文把机器学习算法的自动调参放在贝叶斯优化框架下:把学习算法的泛化性能建模为高斯过程(GP)的一个样本,其可处理的后验使得可以高效利用已有实验信息来选择下一组参数。作者说明 GP 先验和推断流程的选择会显著影响成败,审慎的选择可得到超过专家水平的调参结果;还提出考虑实验耗时变化的新算法,以及利用多核并行实验的算法,在潜在狄利克雷分配、结构化 SVM 和卷积神经网络上达到或超过人类专家的优化水平。
HN 上的人把它当作用贝叶斯优化替代网格搜索来调超参的代表论文,称其在多种模型上效果很好,并提到它对应的 Spearmint 工具和并行实践。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“was a great recent paper on this topic that looked to have very nice results in practice across a range of models.”
“A recent paper used Bayesian optimization with GPs to find hyperparameter settings for a deep convolutional network.”
“I am more familiar with the work of Snoek et al. [1] from NIPS in December last year.”
“[6]: Practical Bayesian Optimization of Machine Learning Algorithms”
“This is the spearmint paper, describes how bayesian search algorithms can be parallelized in practice.”
看全部 7 条 →
“[1] Bayesian Optimization using Gaussian Processes is an example of this. Practical Bayesian Optimization of Machine Learning Algorithms”
- Practical Bayesian Optimization of Machine Learning Algorithms (2012) ↗PaulHoule · 2015-04-22 · 2 分
- PRACTICAL BAYESIAN OPTIMIZATION OF MACHINE LEARNING ALGORITHMS [pdf] ↗mirceasoaica · 2015-05-29 · 1 分