按年被提到
20182018 · 2 次2026
机器学习与深度学习›优化与可微编程·arXiv:1802.03676 · stat.ML

Differentiable Dynamic Programming for Structured Prediction and Attention

Arthur Mensch, Mathieu Blondel · 2018
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2018-05-11 · 最近 2018-11-02
2不同的人
2活跃月份
2讨论串
78被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Dynamic programming (DP) solves a variety of structured combinatorial problems by iteratively breaking them down into smaller subproblems. In spite of their versatility, DP algorithms are usually non-differentiable, which hampers their use as a layer in neural networks trained by backpropagation. To address this issue, we propose to smooth the max operator in the dynamic programming recursion, using a strongly convex regularizer. This allows to relax both the optimal value and solution of the original combinatorial problem, and turns a broad class of DP algorithms into differentiable operators. Theoretically, we provide a new probabilistic perspective on backpropagating through these DP operators, and relate them to inference in graphical models. We derive two particular instantiations of our framework, a smoothed Viterbi algorithm for sequence prediction and a smoothed DTW algorithm for time-series alignment. We showcase these instantiations on two structured prediction tasks and on structured and sparse attention for neural machine translation.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论mxwsn · 2018-11

“Hm, please correct me if I'm wrong, but isn't the Zuker algorithm inherently a dynamic programming algorithm? If so, I would have liked to see this paper compare to directly optimizing the input space using a differentiable form of the Zuker algorithm, e.g., using ideas related to https://arxiv.org/abs/1802.03676. IMO it's unnatural to rely on deep reinf…”

Machine learning spots natural selection at work in human genome原评论 ↗
被提交为 story · 1 次