按年被提到
20242025 · 4 次2026
大模型与语言模型›序列模型与注意力机制·arXiv:2410.01201 · cs.LG

Were RNNs All We Needed?

RNN 就够了吗?
Leo Feng, Frederick Tung, Mohamed Osama Ahmed, Yoshua Bengio, Hossein Hajimirsadeghi · 2024

简化 LSTM 与 GRU,得到可并行训练、性能比肩 Transformer 的 minRNN

开始阅读 ↗PDFalphaXiv 讨论
被推荐 · 7 人首次 2024-10-03 · 最近 2026-02-04
7不同的人
7活跃月份
7讨论串
31被引(OpenAlex)

作者回顾 LSTM(1997)与 GRU(2014),通过简化得到 minLSTM 和 minGRU:参数比原版更少,训练时可完全并行,并在一系列任务上取得与包括 Transformer 在内的近期模型相当的表现。

HN 为什么提它

评论者把它当作循环网络复兴的代表:极简 RNN 能并行训练,说明某些 Transformer 之前的序列模型在合适规模下也能追上 Transformer。

方法难度 3 / 5约 30 分钟经得起时间
读之前最好知道
1LSTM 与 GRU
2Transformer 架构
3序列建模
4反向传播与并行训练

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
推荐inciampati · 2025-03

“this fascinating and popular recent paper on RNNs [1]”

称这篇论文引人入胜,说明深层堆叠使 minGRU 和 minLSTM 能学长依赖
Deep Learning Is Not So Mysterious or Different原评论 ↗
引用xnx · 2024-10

“Interesting work on reviving RNNs. Were RNNs All We Needed?”

认为复兴 RNN 的工作很有意思
Were RNNs all we needed?原评论 ↗
引用cs702 · 2024-12

“I would have thought one of those "minimum viable" RNNs (like Were RNNs All We Needed?) would have been ideal for this.”

认为这类最小可行 RNN 本应最适合当前场景
Byte Latent Transformer: Patches Scale Better Than Tokens原评论 ↗
引用bravura · 2025-01

“how minimal versions of GRUs and LSTMs can be trained in parallel by removing some parameters”

说明去掉部分参数后 GRU 和 LSTM 的精简版可并行训练
RWKV Language Model原评论 ↗
引用rsfern · 2025-07

“finding that some pre-transformer sequence models can match transformers when trained at appropriate scale.”

引它说明部分前 Transformer 序列模型在合适规模下可媲美 Transformer
There are no new ideas in AI, only new datasets原评论 ↗
引用akst · 2025-09

“I think the author may have linked to the wrong paper in their opening paragraph”

指出作者开头段落可能链接了错误的论文,这篇才是可并行训练的简化 LSTM/GRU
Were RNNs all we needed? A GPU programming perspective原评论 ↗
看全部 7 条 →
引用cs702 · 2026-02

“* Linear RNNs: Were RNNs All We Needed?”

列入线性 RNN 相关阅读
FlashAttention-T: Towards Tensorized Attention原评论 ↗
被提交为 story · 1 次