引用taliesinb · 2016-08
“but it does show that feedback weights that aren't computing the actual derivative dloss/din can till support learning.”
拿它说明反馈权重即使不是真实梯度 dloss/din 也能支持学习
用随机固定的反馈权重代替精确反传,网络依然能学会
反向传播要求把下游突触权重精确地传回去,这在大脑中被认为不可能。论文提出一种出人意料地简单的算法:用随机的突触权重去乘误差信号来分配“责任”,网络仍能从这些随机反馈连接中学到有用信息。作者称这一机制在多种问题上与反向传播一样快、一样准,并以此说明神经回路学习的算法约束没有想象的那么严。
评论者用它说明反馈权重并不需要是精确的梯度也能支持学习,并指出这一初步发现发表在这篇预印本里。
“but it does show that feedback weights that aren't computing the actual derivative dloss/din can till support learning.”
“but the initial finding by Lilllicrap et al. has been published as a preprint: Random feedback weights support learning in deep neural networks”