按年被提到
20172017 · 2 次2026
强化学习与智能体›深度强化学习与世界模型·arXiv:1707.06170 · cs.AI

Learning model-based planning from scratch

Razvan Pascanu, Yujia Li, Oriol Vinyals, Nicolas Heess, Lars Buesing, Sebastien Racanière, David Reichert, Théophane Weber, Daan Wierstra, Peter Battaglia · 2017
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2017-07-25 · 最近 2017-08-09
2不同的人
2活跃月份
2讨论串
72被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Conventional wisdom holds that model-based planning is a powerful approach to sequential decision-making. It is often very challenging in practice, however, because while a model can be used to evaluate a plan, it does not prescribe how to construct a plan. Here we introduce the "Imagination-based Planner", the first model-based, sequential decision-making agent that can learn to construct, evaluate, and execute plans. Before any action, it can perform a variable number of imagination steps, which involve proposing an imagined action and evaluating it with its model-based imagination. All imagined actions and outcomes are aggregated, iteratively, into a "plan context" which conditions future real and imagined actions. The agent can even decide how to imagine: testing out alternative imagined actions, chaining sequences of actions together, or building a more complex "imagination tree" by navigating flexibly among the previously imagined states using a learned policy. And our agent can learn to plan economically, jointly optimizing for external rewards and computational costs associated with using its imagination. We show that our architecture can learn to solve a challenging continuous control problem, and also learn elaborate planning strategies in a discrete maze-solving task. Our work opens a new direction toward learning the components of a model-based planning system and how to use them.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论gradstudent · 2017-07

“I'm not a planning guy but I work in a closely related community so I'm a least somewhat familar with the area. Looking at the first paper (https://arxiv.org/pdf/1707.06170.pdf), it seems surprisingly shallow and light on details. So they have a learning system for continuous planning. So what? The AI Planning community has been doing this for ages with …”

Agents that imagine and plan原评论 ↗
评论gwern · 2017-08

“I think it is doable in under 5 years, but this critically depends on the resources invested by DM and other DL orgs. Deep RL is hugely demanding of computational resources to iterate your designs - for example, the first AlphaGo took something like 3 GPU-years to train it once (2 or 3 months parallelized); however, with much more iteration, DM was able t…”

DeepMind and Blizzard Open StarCraft II as an AI Research Environment原评论 ↗