按年被提到
20182019 · 1 次2026
视觉、语音与图像生成›卷积网络与图像识别·arXiv:1805.08974 · cs.CV

Do Better ImageNet Models Transfer Better?

Simon Kornblith, Jonathon Shlens, Quoc V. Le · 2018
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2019-02-14 · 最近 2020-09-24
2不同的人
2活跃月份
2讨论串
1,359被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Transfer learning is a cornerstone of computer vision, yet little work has been done to evaluate the relationship between architecture and transfer. An implicit hypothesis in modern computer vision research is that models that perform better on ImageNet necessarily perform better on other vision tasks. However, this hypothesis has never been systematically tested. Here, we compare the performance of 16 classification networks on 12 image classification datasets. We find that, when networks are used as fixed feature extractors or fine-tuned, there is a strong correlation between ImageNet accuracy and transfer accuracy ($r = 0.99$ and $0.96$, respectively). In the former setting, we find that this relationship is very sensitive to the way in which networks are trained on ImageNet; many common forms of regularization slightly improve ImageNet accuracy but yield penultimate layer features that are much worse for transfer learning. Additionally, we find that, on two small fine-grained image classification datasets, pretraining on ImageNet provides minimal benefits, indicating the learned features from ImageNet do not transfer well to fine-grained tasks. Together, our results show that ImageNet architectures generalize well across datasets, but ImageNet features are less general than previously suggested.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论stared · 2019-02

“A few caveats here: - It works (that well) only for vision (for language it sort-of-works only at the word level: http://p.migdal.pl/2017/01/06/king-man-woman-queen-why.html) - "Do Better ImageNet Models Transfer Better?" https://arxiv.org/abs/1805.08974 And if you want to play with transfer learning, here is a tutorial with a working notebook: https:/…”

The Unreasonable Effectiveness of Deep Feature Extraction原评论 ↗
评论gwern · 2020-09

“Yes, because your validation set performance is imperfectly correlated with performance on more samples (because it's of finite size), so it will tend to overestimate within-distribution. There's also the problem that you probably want to deploy it on data which was not exactly collected the same way. So you have issues of both internal and external valid…”

Why neural networks struggle with the Game of Life原评论 ↗