“this is THE paper to read: Power-law distributions in empirical data (it's an easy read, explaining what the maximum likelihood method is, etc.)”
Power-law distributions in empirical data
用最大似然拟合加拟合优度检验,正确判断数据是否服从幂律
论文指出最小二乘等常用幂律分析方法会给出严重失真的参数估计,也无法回答数据到底是否服从幂律。作者提出一套把最大似然拟合与基于 Kolmogorov-Smirnov 统计量和似然比的拟合优度检验结合起来的统计框架,用合成数据验证其有效性,并检验了 24 个来自不同学科的真实数据集:有些幂律猜想与数据相符,有些则被排除。
HN 上的人在讨论幂律、长尾和 log-log 图上拟合直线的做法时反复引用它,说明正确的做法是最大似然估计加统计检验,并称它写得清楚易读。
学习路径
读这篇之前,按顺序读书架上的 1 篇 · 参考文献来自 OpenAlex,顺序和要点由 Claude Code 排出先从综述了解幂律的经验和成因背景,再读如何用最大似然和拟合优度检验严谨地判断幂律。
- 1
综述各领域的幂律现象及成因,目标论文要纠正的正是这类研究里常见的拟合做法。
读读幂律的基本性质与累积分布、用直方图拟合的常见做法;成因模型可略读。
- 2Power-law distributions in empirical data
给出最大似然拟合加拟合优度检验的严谨流程。
读抓估计 xmin 和指数的方法、KS 统计量、与其他分布的似然比比较。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“Also pay close attention to the content of shalizi and newman's review article”
“Actually, the paper you should be reading is this:”
“For example in their excellent review article Power-law distributions in empirical data”
“You might want to check out the paper for more details.”
“Clauset, Shalizi, and Newman have a very readable paper where they describe 1) how to properly test for a power-law distribution, and 2) use those tests to assess the validity of some claims from the literature”
看全部 12 条 →
“In general, it's much better to do a direct statistical test, although the method described in the paper is somewhat involved.”
“A postdoc who was there wrote a paper many years later about the right way to do it”
“Clauset Shalizi Newman 2007 has not-nice things to say about the classic physicist's idiot trick of fitting power law distributions by drawing a straight line on a log-log graph: it's got huge bias.”
“This paper covers a lot of the frequent problems in academic literature that tries to fit power laws:”
“It is a sore point for me that physicists have long wrongly plotted distributions on log-log paper”
“Referenced in the paper: "Power-law distributions in empirical data" - Power-law distributions in empirical data”