按年被提到
20252025 · 18 次2026
软件工程与 AI 编程›软件工程实证研究·arXiv:2507.09089 · cs.AI

Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity

衡量 2025 年初 AI 对资深开源开发者生产力的影响
Joel Becker, Nate Rush, Elizabeth Barnes, David Rein · 2025

随机对照试验:AI 工具使资深开发者慢了 19%

开始阅读 ↗PDFalphaXiv 讨论
被推荐 · 24 人首次 2025-07-17 · 最近 2026-10-15
24不同的人
9活跃月份
23讨论串
134被引(OpenAlex)

研究让 16 名资深开源开发者在自己平均有 5 年经验的成熟项目里完成 246 个任务,每个任务随机分配为允许或禁止使用 2025 年初的 AI 工具(主要是 Cursor Pro 与 Claude 3.5/3.7 Sonnet)。开发者事前预测 AI 会让耗时减少 24%,事后仍估计减少了 20%,但实测允许使用 AI 反而使完成时间增加了 19%,与经济学和机器学习专家的预测也相反。

HN 为什么提它

评论者反复引用它说明开发者对 AI 提速的主观感受并不可靠,也有人批评样本只有 16 人、对象是早期 AI,结论已经过时。

实证研究难度 2 / 5约 45 分钟有时效性
读之前最好知道
1随机对照试验(RCT)基础
2AI 编程助手(Cursor、Claude)
3软件工程实践
4基础统计推断

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
推荐aix1 · 2025-09

“The study you referred to sounded super interesting, so I looked it up to read later.”

表示被提到的这项研究很有趣,打算查来细读
Nvidia buys $5B in Intel原评论 ↗
批评EnnEmmEss · 2025-07

“this should be studied more in depth over a longer time period since the current study had a 50 hour threshold”

The hype is the product原评论 ↗
批评djoldman · 2025-08

“Folks probably should be skeptical of a study with the combination of a small number of subjects with a broad claim.”

批评meander_water · 2025-10

“The METR paper surveyed just 16 developers to arrive at their conclusion. Not sure how that got past review.”

AI Pullback Has Officially Started原评论 ↗
批评john_strinlai · 2026-10

“it seems a bit silly to rely on a paper about early-2025 ai when people are using late-2026 ai”

批评依赖 2025 年初的 AI 研究很可笑,现在 AI 已大幅进步
Libreboot 26.10 "Ominous Olive" released with UEFI, Chromebook support原评论 ↗
引用shakna · 2025-07

“And that self-assessment is flawed.”

拿它说明开发者的自我评估并不可靠
I avoid using LLMs as a publisher and writer原评论 ↗
看全部 20 条 →
引用almog · 2025-07

“(even when they believe that it does)”

拿它说明 AI 工具可能并不提高生产力,哪怕人们自认如此
Two narratives about AI原评论 ↗
引用HarHarVeryFunny · 2025-08

“There was a recent study concluding that AI made experienced developers 20% SLOWER to complete tasks rather than any faster !”

引用它说明 AI 让资深开发者完成任务慢了 20%
Things that helped me get out of the AI 10x engineer imposter syndrome原评论 ↗
引用raincole · 2025-09

“I can tell this is going to be the most misquoted study in blogs and pop-sci books after the 10,000-hour mastery study. And it's just a preprint!”

预言它会被误引,并提醒这只是预印本
AI coding原评论 ↗
引用dpflan · 2025-09

“The study you are alluding to is this one by METR (Model Evaluation & Threat Research):”

指出对方所说的研究就是 METR 这篇
AI was supposed to help juniors shine. Why does it mostly make seniors stronger?原评论 ↗
引用y0eswddl · 2025-10

“There's literally a study out the shows when developers think LLMs are making them 20% faster”

拿它说明开发者自以为 LLM 提速时,实际并非如此
How the AI Bubble Will Pop原评论 ↗
引用Ianjit · 2025-10

“this is what Metr found in its study of developer productivity.”

引用 METR 的发现:这类任务上 AI 没有节省时间
Why the push for Agentic when models can barely follow a simple instruction?原评论 ↗
引用autoexec · 2025-11

“Unlike higher level programing languages AI doesn't actually make programmers more efficient.”

拿它说明 AI 不像高级语言那样让程序员更高效
Baby Shoggoth Is Listening原评论 ↗
引用recitedropper · 2025-11

“there is explicit evidence it does slow them down, even when they think it sped them up: Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity”

拿它说明 AI 确实拖慢开发者,即使他们觉得变快了
Google Antigravity原评论 ↗
引用lunar_mycroft · 2025-12

“that METR study should have completely killed any illusions that that is a reliable metric”

认为 METR 研究应打破对这种指标可靠性的幻想
If you're going to vibe code, why not do it in C?原评论 ↗
引用yorwba · 2026-01

“The METR study referenced (but not linked) in that paragraph has a breakdown of time spent Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity in figure 19 on page 33 which contradicts the description in the Register article.”

指出其图 19 的时间分布与报道描述相矛盾
OpenSlopware deleted, forked, and revived – by me on El Reg原评论 ↗
引用trashb · 2026-01

“The following paper backs this idea Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity”

拿它佐证感知到的 AI 提速与实际并不一致
Ask HN: Do you have any evidence that agentic coding works?原评论 ↗
引用sockopen · 2026-01

“Surprisingly, we find that allowing AI actually increases completion time by 19%--AI tooling slowed developers down.”

引用其结论:允许使用 AI 反而使完成时间增加 19%
Ask HN: Do you have any evidence that agentic coding works?原评论 ↗
引用Miladyshady · 2026-02

“But I know that using in AI in software engineering reduces productivity by almost 20%.”

拿它说明软件工程中用 AI 会让生产力下降近 20%
Tell HN: 3 months ago we feared AI was useless. Now we fear it will take our job原评论 ↗
引用ghboo0927 · 2026-10

“A double-difference comparison between GitHub projects that have adopted Cursor and paired projects that have not:”

与一项 Cursor 双重差分研究并列引用
I'm sorry, but you still have to think原评论 ↗