按年被提到
20172023 · 34 次2026
大模型与语言模型›序列模型与注意力机制·arXiv:1706.03762 · cs.CL

Attention Is All You Need

注意力就是你所需要的一切
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin · 2017

提出完全基于注意力机制的 Transformer 架构,取代循环与卷积

开始阅读 ↗PDFalphaXiv 讨论
被推荐 · 93 人首次 2017-06-13 · 最近 2026-09-30
93不同的人
48活跃月份
100讨论串
120,404被引(OpenAlex)

论文提出 Transformer,一种只依赖注意力机制、完全不用循环和卷积的序列转换网络。在 WMT 2014 英德和英法机器翻译任务上,它的质量超过当时的最好结果,并且更易并行、训练成本明显更低;它在英语成分句法分析上也表现良好。

HN 为什么提它

它被反复称为现代大语言模型的奠基论文,HN 上的人把它当作理解 Transformer 架构的原始出处和入门起点,也有人说原文其实并不好读。

奠基之作难度 3 / 5约 50 分钟经得起时间
读之前最好知道
1神经网络基础
2序列到序列模型(seq2seq)与循环神经网络(RNN)
3注意力机制(attention)
4机器翻译与 BLEU 指标

学习路径

读这篇之前,按顺序读书架上的 4 篇 · 参考文献来自 OpenAlex,顺序和要点由 Claude Code 排出

先理解序列到序列框架和注意力机制如何解决定长向量瓶颈,再了解残差连接和当时的 LSTM 翻译系统,就能看清 Transformer 要取代的是什么。

  1. 1
    Sequence to Sequence Learning with Neural Networks用神经网络进行序列到序列学习 · 2014

    Transformer 沿用的编码器-解码器序列转换框架来自这里

    读编码器-解码器结构与端到端训练;理解它的定长向量瓶颈

  2. 2
    Neural Machine Translation by Jointly Learning to Align and Translate通过联合学习对齐与翻译实现神经机器翻译 · 2014

    Transformer 的注意力思想源于这里的软对齐

    读第 3 节的对齐模型与上下文向量计算;理解注意力权重的含义

  3. 3
    Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation谷歌神经机器翻译系统:弥合人类与机器翻译之间的差距 · 2016

    Transformer 要超越的 LSTM+注意力的工业级基线

    读8 层 LSTM 编码器-解码器结构与残差连接;对照 Transformer 的 BLEU 和训练成本

  4. 4
    Deep Residual Learning for Image Recognition用于图像识别的深度残差学习 · 2015

    Transformer 每个子层都使用残差连接

    读残差块 y=F(x)+x 的思想与它为何便于训练深层网络;跳过 ImageNet 实验

  5. 5
    Attention Is All You Need注意力就是你所需要的一切 · 2017 · 就是这篇

    目标论文:用纯注意力取代循环和卷积

    读第 3 节的整体架构、缩放点积注意力、多头注意力和位置编码;第 4 节说明为何选自注意力

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
推荐make3 · 2018-04

“An example of this is neural translation with the Attention is all you need paper, which is the top of the line in translation.”

举 Attention is all you need 为例,称其为翻译领域最顶尖的模型
Attention and Augmented Recurrent Neural Networks原评论 ↗
推荐make3 · 2019-11

“like having read the seminal Transformer paper,”

把读过开创性的 Transformer 论文当作读代码的背景知识
OpenAI Releases Largest GPT-2 Text Generation Model原评论 ↗
推荐tiborsaas · 2020-11

“Read the original paper on the architecture: Attention Is All You Need”

推荐读这篇关于该架构的原始论文
I asked GPT-3 for the question to “42”原评论 ↗
推荐lacker · 2020-12

“I think this is the best paper to read to get an understanding of this transformer architecture.”

认为这是理解 Transformer 架构最好的一篇论文
Ask HN: What's the best paper you've read in 2020?原评论 ↗
推荐criticaltinker · 2021-12

“The original paper that introduced the Transformer architecture is quite accessible and outlines a lot of the history and rational for the design [1].”

推荐这篇 Transformer 原始论文,通俗易懂,还交代了设计的来龙去脉
The potential of transformers in reinforcement learning原评论 ↗
推荐qigtyofhp · 2023-02

“Read the intro in the original paper "Attention is all you need paper"”

建议读原始论文的引言
Understanding and Coding the Self-Attention Mechanism of Large Language Models原评论 ↗
看全部 20 条 →
推荐tikkun · 2023-04

“Have you read the 2017 paper Attention is All You Need? It's not a book, but it is long form, a few math formulas but not too many.”

推荐这篇 2017 年论文:篇幅较长但公式不多,可作入门读物
Ask HN: Recommended long-form books about LLMs?原评论 ↗
推荐Garcia98 · 2023-04

“The most notable paper is probably the one that defined the concept of transformers, Attention Is All You Need: Attention Is All You Need”

指出这是定义 Transformer 概念的最重要论文
AI / ML / LLM / Transformer Models Timeline原评论 ↗
推荐seanhunter · 2023-05

“If you want to learn more, here's Vaswani et al 2017 - the seminal paper Attention Is All You Need”

给出 Vaswani 等人 2017 年的开创性论文供深入了解
ChatGPT: It has no inner monologue or meta-cognition原评论 ↗
推荐techbruv · 2023-06

“[0]: The original paper: Attention Is All You Need”

附上原始论文作为参考资料
What Is a Transformer Model?原评论 ↗
推荐cratermoon · 2023-07

“Here's some other works you might never have read, based on their titles.”

把它列入值得读的论文,尽管标题听起来像标题党
You Don't Know Jack about Application Performance原评论 ↗
推荐xianshou · 2023-12

“here are the most helpful resources for understanding transformer basics in particular:”

把它列为理解 Transformer 基础最有帮助的资料之一
Ask HN: Daily practices for building AI/ML skills?原评论 ↗
推荐yorwba · 2025-01

“The abstract Attention Is All You Need explains it well:”

指出摘要就把 Transformer 讲得很清楚
Tensor Product Attention Is All You Need原评论 ↗
推荐tankenmate · 2025-05

“This seminal work described the transformer model that is the basis for almost all LLMs, and is almost certainly the most cited paper on AI even though it was only published in 2017.”

称它是几乎所有 LLM 的基础,也很可能是引用最多的 AI 论文
TransMLA: Multi-head latent attention is all you need原评论 ↗
推荐jpc0 · 2025-07

“you should read, is sparked an entire new AI dawn, it might answer your question”

建议读它,认为它开启了新的 AI 时代,或能回答提问
AI capex is so big that it's affecting economic statistics原评论 ↗
推荐max_ · 2026-03

“I think they need to be looked into more. They just work too well.”

认为 Transformer 效果太好,值得更深入研究
Ask HN: What are the greatest discoveries in the last few years?原评论 ↗
推荐nsagent · 2026-07

“I found the original paper to be excellent, but most everyone during my PhD, including my advisor, found it to be inscrutable”

认为原始论文很出色,但读博时多数人包括导师都觉得难懂
Apple Sues OpenAI for Stealing Trade Secrets to Build AI Hardware原评论 ↗
引用mrplank · 2017-06

“Google Brain outperforms LSTMs with Convolutional Networks in speed and accuracy, seeming to confirm LSTMs are not optimal for NLP at least:”

引它说明 Google Brain 的模型在速度和精度上胜过 LSTM,似乎印证 LSTM 并非 NLP 最优
Exploring LSTMs原评论 ↗
引用albertzeyer · 2017-09

“Googles Transformer was made public in June 2017, in the paper Attention is all you need, Attention Is All You Need”

指出 Google 的 Transformer 于 2017 年 6 月发表于这篇论文
Transformer: A Novel Neural Network Architecture for Language Understanding原评论 ↗
引用grandmczeb · 2018-02

“One of the core operations of the transformer network[1] is a (LxL) x (LxE) matrix multiply”

借 Transformer 说明其核心运算之一是一次大矩阵乘法
DLVM: A modern compiler framework for neural network DSLs原评论 ↗