按年被提到
20072016 · 9 次2026
开源书›机器学习与 AI·No.17·简称 ITILA

Information Theory, Inference, and Learning Algorithms

信息论、推断与学习算法
David J.C. MacKay

把信息论、贝叶斯推断和机器学习放在一起讲的经典教材,文风鲜明

免费阅读 ↗
被推荐 · 65 人首次 2007-10-16 · 最近 2026-08-11官方免费在线(版权保留,作者或出版社免费放出)
65不同的人
65活跃月份
43有推荐的月份
80讨论串

David MacKay 把信息论、编码、贝叶斯推断和神经网络放进同一本书,讲清它们在概率视角下如何互相联系。书中穿插大量习题和个人风格浓厚的讲解,不是干巴巴的定理堆砌。和只讲机器学习算法的教材相比,它更强调概率与压缩的统一视角。作者提供免费 PDF,甚至放出了 LaTeX 源文件。

HN 为什么推荐它

HN 上的人把它当作免费、好读又严谨的教材反复推荐,不少人称它是自己最喜欢的书;也有人说它能让人少一点遇到数据就套神经网络的习惯,并看到机器学习、信息论和统计如何统一起来。

难度 4 / 5约 120 小时
适合谁

想从概率与信息视角理解机器学习的自学者

读之前最好知道
1微积分
2线性代数
3概率论基础

怎么读

由 Claude 根据这本书和 HN 上的说法整理,可能有错
  1. 1先读信息论和概率的开头几章,再决定要不要读后面的编码部分
  2. 2对机器学习感兴趣的读者,可以直接跳到推断和神经网络相关章节
  3. 3认真做书中的习题,作者的提示和评注很有价值
  4. 4配合作者公开的讲课视频一起看,更容易跟上

HN 书评

推荐在前,同类按时间 · 每本最多保留 20 条,统计用全部提及 · 作者本人贴的不算
强烈推荐moultano · 2011-02

“My current favorite, Information Theory Inference and Learning Algorithms, available online for free.:)”

强烈推荐,称是目前最爱的免费数学书
Great Math Books as Recommended by Our Readers原评论 ↗
强烈推荐NathanRice · 2012-04

“By far my favorite book on the subject is conveniently available for free on the internet!”

强烈推荐,称是最爱的贝叶斯统计入门书
How Not To Sort By Average Rating原评论 ↗
推荐dpapathanasiou · 2007-11

“David MacKay wrote a good intro book on Bayesian, neural networks, and related topics: Information Theory, Inference, and Learning Algorithms”

推荐 MacKay 这本入门书,涵盖 Bayesian 和神经网络
How to teach a Bayesian spam filter to play chess原评论 ↗
推荐pcowans · 2007-11

“Information Theory, Inference and Learning Algorithms by David MacKay is excellent, and freely available online.”

推荐这本优秀的免费机器学习入门书
Can anyone recommend good books, articles or essays that introduce machine learning?原评论 ↗
推荐mnemonicsloth · 2009-02

“For more details, you might start with:”

推荐作为信息论和密码学的延伸阅读
Mysterious number 6174原评论 ↗
推荐caffeine · 2009-06

“Book recommendation: David MacKay's (free) book”

推荐这本免费书,以问题为导向统一看待 AI/ML
Ask HN: good math books原评论 ↗
看全部 20 条 →
推荐jonp · 2009-09

“same great clarity of explanation”

称赞它对信息论和学习算法讲得清楚
"Sustainable Energy - Without the Hot Air" by David JC MacKay原评论 ↗
推荐codedivine · 2009-10

“May I also recommend a related book, also a free PDF:”

补充推荐这本相关的免费 PDF,读来有趣
Elements Of Statistical Learning: now free pdf原评论 ↗
推荐benm · 2009-11

“it was Information Theory, Inference, and Learning Algorithms that sparked off a lot of study”

说它激发了自己大量的数学学习
Ask HN: the road to learning useful math原评论 ↗
推荐imurray · 2010-01

“More gentle introduction in this (free to download) text book:”

推荐这本免费教材,更温和地介绍 Gallager codes
Explained: Gallager codes原评论 ↗
推荐ced · 2010-09

“see Wiki or David MacKay's wonderful book for details.”

推荐看它了解熵的细节,称其精彩
Measuring the Entropy of the MOS 6502 CPU原评论 ↗
推荐maurits · 2011-08

“And last, but certainly not least:”

列入机器学习书单,作为压轴推荐
Think Stats: Probability and Statistics for Programmers原评论 ↗
推荐monk_the_dog · 2011-11

“here are some good online resources for machine learning:”

列为学习机器学习的免费 PDF 资源
Online graduate-level machine learning course from CMU's Tom Mitchell原评论 ↗
推荐snippyhollow · 2012-12

“excellent and free book BTW”

称它优秀且免费,第 20、22、33 章讲 k-means 和 EM
List of data science and machine learning resources原评论 ↗
推荐Blahah · 2013-09

“I would add to that two classics with free PDFs available from the authors' websites:”

推荐作为机器学习数学的免费经典
Probabilistic Programming & Bayesian Methods for Hackers原评论 ↗
推荐gtani · 2013-09

“for folks looking to delve into the subject, Barber and McKay's freely available content books are terrific”

推荐想深入 Bayesian 主题的人读这本免费书
Bayesian updating of Probability Distributions原评论 ↗
推荐tehwalrus · 2013-10

“And yet another is David Mackay, who taught me at undergrad:”

提到作者曾教过自己,以压缩为主要例子讲信息论
Gzip + poetry = awesome原评论 ↗
推荐fretless · 2014-01

“You could look at the appendixes in texts by Bishop, MacKay, Barber and Rasmussen/Williams to see what they expect”

建议看它的附录,了解预期的线性代数水平
Mistakes Programmers Make when Starting in Machine Learning原评论 ↗
推荐shoo · 2015-06

“Here are a few tangentially related things that may be of interest:”

列为 Bayesian inference 的相关延伸阅读
Bayes’ Theorem – What is it and what is it good for?原评论 ↗
推荐tfgg · 2016-03

“If you have time, I'd also read something like David Mackay's information theory textbook [1] for more of a Bayesian perspective.”

建议读它获得 Bayesian 视角看基础学习结论
Enrollment Is Surging in Machine Learning Classes原评论 ↗