“An interesting idea. A paper from last year showed that neural networks and rational functions efficiently approximate one another [1]. [1] https://arxiv.org/abs/1706.03301”
Neural networks and rational functions
Neural networks and rational functions efficiently approximate each other. In more detail, it is shown here that for any ReLU network, there exists a rational function of degree $O(\text{polylog}(1/ε))$ which is $ε$-close, and similarly for any rational function there exists a ReLU network of size $O(\text{polylog}(1/ε))$ which is $ε$-close. By contrast, polynomials need degree $Ω(\text{poly}(1/ε))$ to approximate even a single ReLU. When converting a ReLU network to a rational function as above, the hidden constants depend exponentially on the number of layers, which is shown to be tight; in other words, a compositional representation can be beneficial even for rational functions.
候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“I would be less surprised if the paper claimed rational functions instead of polynomials. NN's are ~general non-linear functions which polynomials are very poor at representing (e.g. you can't encode asymptotic behavior in polynomials). edit: here's a paper about rational functions and NN's: https://arxiv.org/abs/1706.03301. I'm not really in a position…”