按年被提到
20152015 · 2 次2026
强化学习与智能体›博弈与游戏 AI·arXiv:1509.06461 · cs.LG

Deep Reinforcement Learning with Double Q-learning

Hado van Hasselt, Arthur Guez, David Silver · 2015
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 3 人首次 2015-09-23 · 最近 2025-06-15
3不同的人
2活跃月份
3讨论串
3,542被引(OpenAlex)
arXiv 摘要(候选层不写导读)

The popular Q-learning algorithm is known to overestimate action values under certain conditions. It was not previously known whether, in practice, such overestimations are common, whether they harm performance, and whether they can generally be prevented. In this paper, we answer all these questions affirmatively. In particular, we first show that the recent DQN algorithm, which combines Q-learning with a deep neural network, suffers from substantial overestimations in some games in the Atari 2600 domain. We then show that the idea behind the Double Q-learning algorithm, which was introduced in a tabular setting, can be generalized to work with large-scale function approximation. We propose a specific adaptation to the DQN algorithm and show that the resulting algorithm not only reduces the observed overestimations, as hypothesized, but that this also leads to much better performance on several games.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论ClintEhrlich · 2015-09

“Here is a link to the underlying study, for people who are interested in digging into the data themselves: http://arxiv.org/pdf/1509.06461v1.pdf”

Google AI beats humans at more classic arcade games than ever before原评论 ↗
评论elchananHaas · 2025-06

“I think the article's analysis of overapproximation bias is correct. The issue is that due to the Max operator in the Q learning noise is amplified over timesteps. Some methods to reduce this bias, such as https://arxiv.org/abs/1509.06461 were successful in improving the RL agents performance. Studies have found that this happens even more for the states …”

Q-learning is not yet scalable原评论 ↗
被提交为 story · 1 次