“Similar discussion with a potential solution, in the domain of formal language learning: Bridging the Empirical-Theoretical Gap in Neural Network Formal Language Learning Using Minimum Description Length https://arxiv.org/abs/2402.10013”
Bridging the Empirical-Theoretical Gap in Neural Network Formal Language Learning Using Minimum Description Length
Neural networks offer good approximation to many tasks but consistently fail to reach perfect generalization, even when theoretical work shows that such perfect solutions can be expressed by certain architectures. Using the task of formal language learning, we focus on one simple formal language and show that the theoretically correct solution is in fact not an optimum of commonly used objectives -- even with regularization techniques that according to common wisdom should lead to simple weights and good generalization (L1, L2) or other meta-heuristics (early-stopping, dropout). On the other hand, replacing standard targets with the Minimum Description Length objective (MDL) results in the correct solution being an optimum.
候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及- Bridging the Empirical-Theoretical Gap in Formal Language Learning Using MDL ↗bottencat · 2024-02-17 · 1 分
- Bridging empirical-theoretical gap in neural network formal language learning ↗puttycat · 2024-02-22 · 68 分