引用gwern · 2022-05
“DT is so general it's almost trivial to make it do multi-agent RL as well [OTHER_PAPER] Multi-Agent Reinforcement Learning is a Sequence Modeling Problem”
拿它说明 Decision Transformer 很通用,做多智能体 RL 也很容易
用编码器-解码器 Transformer 把多智能体决策转成序列预测
论文提出 Multi-Agent Transformer(MAT),把合作式多智能体强化学习转化为序列建模问题:把智能体的观测序列映射为最优动作序列。其编码器-解码器结构借助多智能体优势分解定理,把联合策略搜索变成序列决策,只需线性时间复杂度并带有单调提升保证。MAT 通过在线、同策略方式训练,在 StarCraftII、Multi-Agent MuJoCo、灵巧手操作和 Google Research Football 上优于 MAPPO、HAPPO,并在未见任务上表现为出色的少样本学习者。
评论者用它说明 Decision Transformer 这类序列模型非常通用,连多智能体强化学习也能直接套用,即 Transformer 可以做强化学习。
“DT is so general it's almost trivial to make it do multi-agent RL as well [OTHER_PAPER] Multi-Agent Reinforcement Learning is a Sequence Modeling Problem”
“Transformers can do Reinforcement Learning yes.”