按年被提到
20192019 · 4 次2026
机器学习与深度学习›训练加速与低精度计算·arXiv:1908.01580 · cs.LG

The HSIC Bottleneck: Deep Learning without Back-Propagation

Wan-Duo Kurt Ma, J. P. Lewis, W. Bastiaan Kleijn · 2019
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 4 人首次 2019-08-12 · 最近 2024-01-30
4不同的人
2活跃月份
5讨论串
141被引(OpenAlex)
arXiv 摘要(候选层不写导读)

We introduce the HSIC (Hilbert-Schmidt independence criterion) bottleneck for training deep neural networks. The HSIC bottleneck is an alternative to the conventional cross-entropy loss and backpropagation that has a number of distinct advantages. It mitigates exploding and vanishing gradients, resulting in the ability to learn very deep networks without skip connections. There is no requirement for symmetric feedback or update locking. We find that the HSIC bottleneck provides performance on MNIST/FashionMNIST/CIFAR10 classification comparable to backpropagation with a cross-entropy target, even when the system is not encouraged to make the output resemble the classification labels. Appending a single layer trained with SGD (without backpropagation) to reformat the information further improves performance.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论marmaduke · 2019-08

“Mentions a so called HSIC bottleneck, maximizing mutual information between hidden states and labels, minimizing mutual dependence between input and hidden states (?) PDF here https://arxiv.org/pdf/1908.01580v1.pdf Another place I recently saw Reproducing Kernel Hilbert Spaces was in Stein Variational Gradient Descent (PyMC3 implements it). It seems li…”

Does Deep Learning Still Need Backpropagation?原评论 ↗
评论andreyk · 2019-08

“It's good to be wary of hype, but this comment seems overly down on where we are by focusing on AI today doing 'curve fitting'; sure, supervised learning is at it's core that, but when it achieves things like image captioning or super accurate translation calling it just curve fitting is reductionist and downplays how powerful these techniques are. Furt…”

AI startup claims to automate app making but actually just uses humans原评论 ↗
被提交为 story · 3 次