“The arxiv has a more recent one: The Forward-Forward Algorithm”
The Forward-Forward Algorithm: Some Preliminary Investigations
用两次前向传播取代反向传播,每层以“好度”为目标训练网络。
Hinton 提出一种新的神经网络学习方法,用两次前向传播取代反向传播的前向和后向传播:一次输入真实(正)数据,一次输入负数据(可由网络自己生成)。每一层有自己的目标:对正数据有高“好度”(goodness),对负数据有低好度,好度可以是该层激活的平方和。论文说明它在几个小问题上效果足够好,值得进一步研究;若正负传播能在时间上分开,视频可以流水线式通过网络而无需存储激活或停下来反传梯度。
评论者多把它当作 Hinton 在反向传播之外探索的替代学习范式:提到它可能更接近大脑的学习方式,可能用于 LoRA 式的附加层,以及与硬件绑定的“mortal computation”的联系。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“i wonder if its possible to do similar with "The Forward-Forward Algorithm" by Geoff Hilton maybe on a few add on layers like with lora”
“If Hinton really, genuinely believes the world is ending, then why is he working on the generational successor to backpropagation [1]?”
“that would have overcome this (if it worked) before he decided humanity couldn't be trusted”
“The Forward-Forward Algorithm: Some Preliminary Investigations The Forward-Forward Algorithm”
“Hinton has some research on a forward-forward learning paradigm [1], which might be closer to how our brains learn (but the artificial implementations are not great yet).”
- The Forward-Forward Algorithm, potential alternative to backpropagation ↗mfi · 2023-01-04 · 1 分
- The Forward-Forward Algorithm: Some Preliminary Investigations (2022) ↗lnyan · 2023-05-02 · 1 分