“The author states in his github repository (https://github.com/MarkPKCollier/NeuralTuringMachine) that his work is based on (https://github.com/snowkylin/ntm). If that is the case then I find it kind of strange that he has relicensed the work to MIT from LGPL3 and removed any reference to the original author in the LICENSE. LGPL3 requires any derivative w…”
Implementing Neural Turing Machines
Neural Turing Machines (NTMs) are an instance of Memory Augmented Neural Networks, a new class of recurrent neural networks which decouple computation from memory by introducing an external memory unit. NTMs have demonstrated superior performance over Long Short-Term Memory Cells in several sequence learning tasks. A number of open source implementations of NTMs exist but are unstable during training and/or fail to replicate the reported performance of NTMs. This paper presents the details of our successful implementation of a NTM. Our implementation learns to solve three sequential learning tasks from the original NTM paper. We find that the choice of memory contents initialization scheme is crucial in successfully implementing a NTM. Networks with memory contents initialized to small constant values converge on average 2 times faster than the next best memory contents initialization scheme.
候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“I know this is a big issue in AI/ML right now. Deepmind's papers are notoriously hard to reproduce, because they will lay out the general terms of the architecture but not specific implementation details - things like filter length, stride, number of layers, number of hidden units, feature selection, and all the little tricks of initialization or normaliz…”
- Implementing Neural Turing Machines ↗godelmachine · 2018-07-25 · 117 分