“>One of the domains where I hope Julia will excel is precisely as you comment on physics + ml (I check frequently if MuJoCo source code is available at last). Somebody already did a comparison of DifferentialEquations.jl's standard usage against MuJoCo and DiffTaichi and showed that it outperformed them by about 5x-10x. https://arxiv.org/abs/2012.06684 …”
Faster Policy Learning with Continuous-Time Gradients
We study the estimation of policy gradients for continuous-time systems with known dynamics. By reframing policy learning in continuous-time, we show that it is possible construct a more efficient and accurate gradient estimator. The standard back-propagation through time estimator (BPTT) computes exact gradients for a crude discretization of the continuous-time system. In contrast, we approximate continuous-time gradients in the original system. With the explicit goal of estimating continuous-time gradients, we are able to discretize adaptively and construct a more efficient policy gradient estimator which we call the Continuous-Time Policy Gradient (CTPG). We show that replacing BPTT policy gradients with more efficient CTPG estimates results in faster and more robust learning in a variety of control tasks and simulators.
候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“I manage the MuJoCo.jl wrapper, and having the source code wont (naively) help with AD. Internally, there's a number of iterative algorithms that you wouldn't want to automatically differentiate, and if analytical derivatives were apparent they would be in there (the MuJoCo creator was my advisor). As it stands, finite differencing is the best way to extr…”
“The only ones that I know of are in https://arxiv.org/abs/2012.06684 (with Julia DifferentialEquations.jl and DiffTaichi), but those are more algorithmic. There, Julia does extremely well, but the conclusion that I would draw from it is more: use a programming language with robust and differentiable differential equation solvers rather than writing simple…”