批评aspenmartin · 2026-01
“The Impact of AI on Developer Productivity -- 55% faster implementing HTTP server in javascript with copilot (in 2023!) but this is a single task and not really representative.”
对照实验:用 Copilot 的开发者写 HTTP 服务器快 55.8%。
论文用对照实验检验 GitHub Copilot 对开发者效率的影响:招募的软件开发者被要求尽快用 JavaScript 实现一个 HTTP 服务器。能使用 AI 结对编程工具的实验组比对照组快 55.8% 完成任务。观察到的异质性效应表明,AI 结对编程工具有望帮助更多人转入软件开发职业。
评论者把它当作 AI 编程工具提升效率的主要证据引用(约快 55%),也有人指出它只测了一个任务,代表性有限。
“The Impact of AI on Developer Productivity -- 55% faster implementing HTTP server in javascript with copilot (in 2023!) but this is a single task and not really representative.”
“The CoPilot productivity study by Peng et al. that is summarized in a Blog post [1] with some additional details in the Arxiv preprint [2].”
“A joint Harvard / MIT study found developers completed tasks up to 50% faster [0]!”
“here are a couple I know of offhand, showing a 25% to 55% boost.”