“Silver et al - Mastering Chess and Shogi [4]”
Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm
AlphaZero:同一套自我对弈强化学习,24 小时内称霸国际象棋、将棋和围棋
把 AlphaGo Zero 的自我对弈方法推广成单一的 AlphaZero 算法:从随机下法开始,除规则外不使用任何领域知识,在 24 小时内就在国际象棋、将棋和围棋上达到超人水平,并分别令人信服地击败了该项目的世界冠军程序。
HN 上的人把它当作一套通用自我对弈强化学习即可掌握多种棋类的出处,常被拿来核对细节,例如与 Stockfish 的比赛设定、超参数调节和走法编码。
学习路径
读这篇之前,按顺序读书架上的 5 篇 · 参考文献来自 OpenAlex,顺序和要点由 Claude Code 排出从深度 RL 起点 DQN 出发,看用 CNN 评估围棋着法、神经网络下棋的早期尝试,以及神经网络加树搜索的专家迭代思想,最后读 AlphaZero。
- 1
深度 RL 的基础,AlphaZero 的价值学习建立在这类思想之上。
读Q 学习和用卷积网络处理原始输入的设定。
- 2
用深度 CNN 评估围棋着法,是神经网络进入围棋的早期工作,对应 AlphaZero 的策略网络。
读把棋盘当图像、预测下一手的思路。
- 3
同期用卷积网络预测专家落子,不用搜索也有不错水平,说明策略网络的价值。
读监督学习预测专家着法,以及它与树搜索结合的动机。
- 4
Giraffe 用自我对弈训练评估函数下国际象棋,与 AlphaZero 的方向一致但更朴素。
读神经网络评估函数加自我对弈的训练流程,对比 AlphaZero 的通用性。
- 5
专家迭代:用树搜索产生训练目标来改进神经网络,AlphaZero 的训练循环同理。
读搜索作为专家、网络作为学徒的迭代结构。
- 6Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm
目标论文:用同一套自我对弈强化学习,通用地学会国际象棋、将棋和围棋。
读网络加 MCTS 的自我对弈循环、训练目标和与 AlphaGo Zero 的差别、与传统引擎的比较。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“I thought the AlphaZero paper was pretty cool: Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm”
“Here's an HTML version of the paper:”
“while it doesn't explicitly state whether Stockfish was allowed to use them in the tests”
“AlphaZero tackled structured games with a single algorithm [1].”
“The paper Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm doesn't go into detail about hyperparameter tuning, but does say that they used Bayesian optimization.”
看全部 11 条 →
“The information you want is on page 13, last paragraph.”
“The results were published - Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm”
“the abstract of the AlphaZero paper [0] seems to disagree about your /any prior knowledge/ point:”
“The paper: Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm”
“By contrast, the AlphaGo [0] AlphaZero [1] and AlphaStar [2] papers claim "mastery", "superhuman", "world champion level", "Grandmaster-level", "human professional" ability - all defensible claims given their performance and match conditions in the respective games.”
- Mastering Chess and Shogi by Self-Play with General Reinforcement Learning ↗dennybritz · 2017-12-06 · 539 分