按年被提到
20132015 · 4 次2026
大模型与语言模型›序列模型与注意力机制·arXiv:1308.0850 · cs.NE

Generating Sequences With Recurrent Neural Networks

用循环神经网络生成序列
Alex Graves · 2013

用 LSTM 逐点预测生成文本,并合成逼真的手写体

开始阅读 ↗PDFalphaXiv 讨论
被推荐 · 11 人首次 2015-06-11 · 最近 2025-02-08
11不同的人
12活跃月份
11讨论串
2,890被引(OpenAlex)

论文展示了如何仅通过每次预测一个数据点,用长短期记忆(LSTM)循环神经网络生成具有长程结构的复杂序列,并在文本(离散数据)和在线手写(实值数据)上演示。作者进一步让网络的预测以文本序列为条件,实现手写合成,所得系统能生成各种风格、高度逼真的草书手写体。

HN 为什么提它

HN 上的人把它当作 LSTM 序列生成的早期经典,称它启发了后来大量手写和语音合成工作,并引用其中 RNN 生成维基百科文本等样例,也说其技术可移植到任何现代框架。

方法难度 3 / 5约 120 分钟经得起时间
读之前最好知道
1循环神经网络(RNN)与 LSTM
2自回归(autoregressive)序列建模
3混合密度网络(mixture density network)
4条件生成与注意力式对齐的基本思想

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
推荐kastnerkyle · 2015-06

“No official publication to date on the speech experiments themselves, but the procedure is well documented in his paper [3]”

指出语音实验尚无正式发表,但其方法在这篇论文里有详细记录
Extending “Let It Go” with recurrent neural network原评论 ↗
推荐nrmn · 2015-12

“Older one, but important to understand deeply since other recent ideas have come from this!”

称这篇虽老但很重要,需要深入理解,后来许多想法源自它
Ask HN: Papers you read in 2015?原评论 ↗
推荐twotwotwo · 2016-05

“Man, the RNN-generated Wikipedia data around page 14 of Generating Sequences With Recurrent Neural Networks is fascinating.”

觉得第 14 页左右 RNN 生成的维基百科文本很迷人
Neural Networks Are Impressively Good at Compression原评论 ↗
推荐dharma1 · 2025-02

“But one of the early deep learning papers from Alex Graves does this really well with LSTMs - Generating Sequences With Recurrent Neural Networks”

指出 Graves 早期深度学习论文用 LSTM 把这件事做得很好
Ghostwriter – use the reMarkable2 as an interface to vision-LLMs原评论 ↗
引用kastnerkyle · 2015-08

“handwriting generation [12]”

把手写生成列为循环网络的众多应用之一
A Beginner’s Guide to Restricted Boltzmann Machines原评论 ↗
引用yankoff · 2015-12

“Generating Sequences With Recurrent Neural Networks”

把它列入 2015 年读过的论文清单
Ask HN: Papers you read in 2015?原评论 ↗
看全部 15 条 →
引用kastnerkyle · 2016-03

“Other attentions such as [2],[3] are constrained either to only move forward or to be a hybrid of forward movement and (pooled) global lookup.”

拿它作对照:只能向前移动的 attention 变体
Dynamic Memory Networks for Visual and Textual Question Answering原评论 ↗
引用walrus · 2016-04

“Alex Graves achieved 1.33 bits per character using stacked LSTMs.[1]”

引用 Graves 用堆叠 LSTM 达到每字符 1.33 比特
Neural Networks Are Impressively Good at Compression原评论 ↗
引用tgflynn · 2016-05

“some of the most impressive deep learning results have been obtained with these.”

指出一些最惊人的深度学习成果就是用这类网络取得的
Neural Networks Are Impressively Good at Compression原评论 ↗
引用kastnerkyle · 2017-04

“There is an older paper [0] and demo from [1] Alex Graves that inspired a ton of work around handwriting, and then speech.”

提到 Graves 这篇早期论文,启发了手写和语音方面的大量工作
Lyrebird – An API to copy the voice of anyone原评论 ↗
引用hardmaru · 2018-04

“We took the terms from existing works that we knew:”

说术语取自已有工作,其中把 RNN 序列生成比作做梦
A step-by-step guide to the “World Models” AI paper原评论 ↗
引用kastnerkyle · 2018-11

“work on "Generating Sequences With Recurrent Neural Networks", including his unpublished TTS demo [1].”

指出用拼接方式做序列条件化的源头在 Graves 的 RNN 工作
Varying Speaking Styles with Neural Text-to-Speech原评论 ↗
引用sjv · 2020-08

“This problem was more or less solved with neural networks in 2013, see Generating Sequences With Recurrent Neural Networks.”

指出这个问题在 2013 年已基本被神经网络解决
An attempt to make a font look more handwritten原评论 ↗
引用duckmysick · 2023-01

“Looks like it's based on this paper [1].”

指出这个演示基于这篇论文
Realistic Computer-Generated Handwriting原评论 ↗
引用hm64 · 2023-01

“One could also port the techniques described in the paper, Generating Sequences With Recurrent Neural Networks using any modern machine learning framework.”

指出论文中的技术可用任何现代机器学习框架移植
Realistic Computer-Generated Handwriting原评论 ↗