按年被提到
20212023 · 1 次2026
机器学习与深度学习›神经科学启发的 AI·arXiv:2106.13898 · cs.LG

Closed-form Continuous-time Neural Models

Ramin Hasani, Mathias Lechner, Alexander Amini, Lucas Liebenwein, Aaron Ray, Max Tschaikowski, Gerald Teschl, Daniela Rus · 2021
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2023-02-08 · 最近 2024-09-30
2不同的人
2活跃月份
2讨论串
236被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Continuous-time neural processes are performant sequential decision-makers that are built by differential equations (DE). However, their expressive power when they are deployed on computers is bottlenecked by numerical DE solvers. This limitation has significantly slowed down the scaling and understanding of numerous natural physical phenomena such as the dynamics of nervous systems. Ideally, we would circumvent this bottleneck by solving the given dynamical system in closed form. This is known to be intractable in general. Here, we show it is possible to closely approximate the interaction between neurons and synapses -- the building blocks of natural and artificial neural networks -- constructed by liquid time-constant networks (LTCs) efficiently in closed-form. To this end, we compute a tightly-bounded approximation of the solution of an integral appearing in LTCs' dynamics, that has had no known closed-form solution so far. This closed-form solution substantially impacts the design of continuous-time and continuous-depth neural models; for instance, since time appears explicitly in closed-form, the formulation relaxes the need for complex numerical solvers. Consequently, we obtain models that are between one and five orders of magnitude faster in training and inference compared to differential equation-based counterparts. More importantly, in contrast to ODE-based continuous networks, closed-form networks can scale remarkably well compared to other deep learning instances. Lastly, as these models are derived from liquid networks, they show remarkable performance in time series modeling, compared to advanced recurrent models.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论f_devd · 2023-02

“Although the article is recent the paper from the article has been available on preprint/arxiv since June 2021[1], implementations for pytorch & tensorflow are also available[2] for those interested. [1]: https://arxiv.org/abs/2106.13898 [2]: https://github.com/raminmh/CfC”

New neural network architecture inspired by neural system of a worm原评论 ↗
评论programjames · 2024-09

“The ideas come from these papers: 1. [Liquid Time-Constant Networks (2020)](https://arxiv.org/abs/2006.04439) This is essentially a neural ODE applied to leaky integrate-and-fire. 2. [Closed-form Continuous-time (2022)](https://arxiv.org/abs/2106.13898) A closed-form approximation of the first.”

Liquid Foundation Models: Our First Series of Generative AI Models原评论 ↗