“some new developments such as dropout can be interpreted as some form of deep Gaussian processes [3]”
Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
把 dropout 训练视为深度高斯过程的近似贝叶斯推断,借此估计不确定性。
深度学习的回归和分类工具不能刻画模型不确定性,而贝叶斯模型有严谨框架却计算代价过高。论文提出一个新的理论框架,把深度神经网络中的 dropout 训练视为深度高斯过程中的近似贝叶斯推断,由此得到利用 dropout 网络建模不确定性的工具,从已有模型中提取此前被丢弃的信息,而不牺牲计算复杂度或测试精度。作者系统研究了 dropout 不确定性的性质,在 MNIST 上评估多种网络结构和非线性,预测对数似然和 RMSE 相比现有最优方法有明显改进,并把它用于深度强化学习。
评论者引用它说明 dropout 在数学上近似于深度高斯过程,因此可以在推理时使用 dropout(Monte Carlo dropout)来近似贝叶斯的模型不确定性。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“is mathematically equivalent to an approximation to the probabilistic deep Gaussian process”
“The only paper I've seen that theoretically analyzes dropout is: Dropout as a Bayesian Approximation, which proves it's equivalent to approximating gaussian processes”
“how does it differ from using drop-out's to approximate Bayesian behavior ala Dropout as a Bayesian Approximation”
“A colleague has given me the idea of applying the Monte Carlo Dropout technique [1] (A interpretation of Dropout as a GP) to the CEAL framework [0].”
“e.g. by using dropout at inference time [0]”