“The most impressive piece of work I'm aware of in this area is LIME”
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
LIME:在预测点附近拟合可解释的局部模型,解释任意分类器
提出 LIME,通过在单个预测附近学习一个可解释的局部模型,以可解释且忠实的方式解释任意分类器的预测;并把「挑选有代表性的个体预测及其解释」建模为子模优化问题(SP-LIME)。作者在文本和图像分类上,通过模拟实验和人类实验,展示了解释在判断是否信任预测、选择模型、改进模型等场景中的作用。
HN 上的人把 LIME 当作可解释机器学习的代表方法,有人提到它在欧盟被用于解释信用决策,也有人用其中的例子说明分类器可能因无关原因而准确。
学习路径
读这篇之前,按顺序读书架上的 2 篇 · 参考文献来自 OpenAlex,顺序和要点由 Claude Code 排出候选里与 LIME 直接相关的不多,主要是理解卷积网络内部可视化与图像分类器(LIME 在图像上解释的对象)。
- 1
代表另一类解释思路:直接可视化卷积网络内部特征,与 LIME 的黑盒局部解释形成对照。
读反卷积可视化的做法与它依赖模型内部结构这一点。
- 2
LIME 的图像实验解释的就是 Inception 这类深度卷积网络的预测。
读只需了解 Inception 是怎样的图像分类网络;不必细读结构细节。
- 3"Why Should I Trust You?": Explaining the Predictions of Any Classifier
目标论文:在预测点附近扰动输入,拟合可解释的局部线性模型。
读局部保真度与可解释性的权衡目标函数,扰动采样与稀疏线性模型;再看 SP-LIME 选代表性样本。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“Found a good resource which explained the trust component:”
“If any one wants the direct link to the arXiv paper, it's here: "Why Should I Trust You?"”
“This paper ([0]) is actually mentioned in the DARPA BAA ([1]) as an example of a possible direction.”
“Roughly speaking, LIME learns a locally approximate model which can be interpreted.”
“Look at section 3.5 where classifier was predicting with 94% accuracy for completely unrelated reasons like a person's name.”
看全部 12 条 →
“LIME is being used within the EU to explain credit decisions and fraud detection flagging on neural network based models, which is quite a high bar to regulatory oversight to pass.”
“explainability of machine learning models is a hot field:”
“The lime [2] package, based on this paper [3], is also notable.”
“A simple counter-example.”
“See also Section 6.3 of this paper for how interpretability can help choose models with superior generalization”
“If they've trained an interpretable student [2]”
- [pdf] Explaining the Predictions of Any Classifier ↗__Joker · 2016-04-12 · 1 分
- “Why Should I Trust You?” Explaining the Predictions of Any Classifier [pdf] ↗kawera · 2016-09-04 · 3 分
- Explaining black box models with LIME ↗campac · 2020-10-20 · 1 分
- “Why Should I Trust You?”: Explaining the Predictions of Any Classifier ↗Cas9 · 2021-07-30 · 1 分