推荐ShamelessC · 2022-01
“Pretty cool stuff. Refreshing to see symbolic problems being tackled with deep learning.”
赞这项工作,很高兴看到深度学习用于符号问题
训练 Transformer 从整数或浮点序列推断其递推关系
训练 Transformer 去推断整数或浮点数序列背后的函数或递推关系,这类任务常见于人类智商测试,却很少被机器学习研究触及。整数模型在 OEIS 序列的一个子集上优于 Mathematica 内置的递推预测函数;浮点模型能为 bessel0(x) 和 1.644934(π²/6)这类函数与常数给出有信息量的近似。
评论者认为它表明 Transformer 能学会数学任务、符号问题也能用深度学习攻克,并指出其数字的 token 化方式有助于泛化到没见过的数。
“Pretty cool stuff. Refreshing to see symbolic problems being tackled with deep learning.”
“Take a look at this paper:”
“Both of these models can generalize to numbers it have not seen.”
“Transformers are certainly capable of learning mathematical tasks; consider [1] as an example, which uses big but regular token lengths.”