按年被提到
20212021 · 1 次2026
机器学习与深度学习›优化与可微编程·arXiv:2105.15183 · cs.LG

Efficient and Modular Implicit Differentiation

Mathieu Blondel, Quentin Berthet, Marco Cuturi, Roy Frostig, Stephan Hoyer, Felipe Llinares-López, Fabian Pedregosa, Jean-Philippe Vert · 2021
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2021-06-03 · 最近 2022-12-11
2不同的人
2活跃月份
2讨论串
94被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Automatic differentiation (autodiff) has revolutionized machine learning. It allows to express complex computations by composing elementary ones in creative ways and removes the burden of computing their derivatives by hand. More recently, differentiation of optimization problem solutions has attracted widespread attention with applications such as optimization layers, and in bi-level problems such as hyper-parameter optimization and meta-learning. However, so far, implicit differentiation remained difficult to use for practitioners, as it often required case-by-case tedious mathematical derivations and implementations. In this paper, we propose automatic implicit differentiation, an efficient and modular approach for implicit differentiation of optimization problems. In our approach, the user defines directly in Python a function $F$ capturing the optimality conditions of the problem to be differentiated. Once this is done, we leverage autodiff of $F$ and the implicit function theorem to automatically differentiate the optimization problem. Our approach thus combines the benefits of implicit differentiation and autodiff. It is efficient as it can be added on top of any state-of-the-art solver and modular as the optimality condition specification is decoupled from the implicit differentiation mechanism. We show that seemingly simple principles allow to recover many existing implicit differentiation methods and create new ones easily. We demonstrate the ease of formulating and solving bi-level optimization problems using our framework. We also showcase an application to the sensitivity analysis of molecular dynamics.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论shoyer · 2021-06

“The general case of implicit differentiation, i.e., for functions y(x) defined by the constraints F(x, y(x)) = 0, where x and y are vectors, is solved by "implicit function theorem": https://en.wikipedia.org/wiki/Implicit_function_theorem ∂ y(x) = -(∂_1 F(x, y(x)))^{-1} (∂_0 F(x, y(x))) where ∂ denotes partial differentiation. This turns out to be an i…”

Implicit Differentiation原评论 ↗
评论jdeaton · 2022-12

“Of course because of criteria 0, 1, 2 these entirely depend on the individual. However, some papers which fit the criteria for me at the time were the following: - Score-Based Generative Modeling through Stochastic Differential Equations https://arxiv.org/abs/2011.13456 - Structured Denoising Diffusion Models in Discrete State-Spaces https://arxiv.org/a…”

Ask HN: How to get back into AI?原评论 ↗