“Mentions a so called HSIC bottleneck, maximizing mutual information between hidden states and labels, minimizing mutual dependence between input and hidden states (?) PDF here https://arxiv.org/pdf/1908.01580v1.pdf Another place I recently saw Reproducing Kernel Hilbert Spaces was in Stein Variational Gradient Descent (PyMC3 implements it). It seems li…”
The HSIC Bottleneck: Deep Learning without Back-Propagation
We introduce the HSIC (Hilbert-Schmidt independence criterion) bottleneck for training deep neural networks. The HSIC bottleneck is an alternative to the conventional cross-entropy loss and backpropagation that has a number of distinct advantages. It mitigates exploding and vanishing gradients, resulting in the ability to learn very deep networks without skip connections. There is no requirement for symmetric feedback or update locking. We find that the HSIC bottleneck provides performance on MNIST/FashionMNIST/CIFAR10 classification comparable to backpropagation with a cross-entropy target, even when the system is not encouraged to make the output resemble the classification labels. Appending a single layer trained with SGD (without backpropagation) to reformat the information further improves performance.
候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“It's good to be wary of hype, but this comment seems overly down on where we are by focusing on AI today doing 'curve fitting'; sure, supervised learning is at it's core that, but when it achieves things like image captioning or super accurate translation calling it just curve fitting is reductionist and downplays how powerful these techniques are. Furt…”
- The HSIC Bottleneck: Deep Learning Without Back-Propagation ↗etiam · 2019-08-12 · 1 分
- The HSIC Bottleneck: Deep Learning Without Back-Propagation ↗etiam · 2019-08-15 · 2 分
- The HSIC Bottleneck: Deep Learning Without Back-Propagation ↗danny00 · 2024-01-30 · 1 分