“There is actually some evidence that coding agents struggle the same way humans do as the complexity of the system increases [0].”
SlopCodeBench: Benchmarking How Coding Agents Degrade Over Long-Horizon Iterative Tasks
迭代扩展任务下,编程智能体的代码如何越写越烂的基准
论文提出 SlopCodeBench,包含 36 个问题、196 个检查点,让智能体反复扩展自己写的解决方案,并度量结构侵蚀与冗余两种退化。在 15 个编程智能体上,没有一个能端到端解决任何问题,最好的只通过 14.8% 的检查点;约 77% 的轨迹结构侵蚀上升、75.5% 的冗余上升,智能体代码比 473 个开源 Python 仓库更冗长、更侵蚀。明确的质量提示能降低初始冗余和侵蚀,但不改变退化速度。
评论者拿它当证据,说明编程智能体随系统复杂度上升和多轮迭代会让代码越来越糟,也有人直接用它跑评测。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“Research Paper — SlopCodeBench”
“here is a similar, more recent research paper from MIT and co:”
“In fact, there are some papers (that we cited) which create a set of tasks doing exactly this, and it is non-trivial [1].”
“Actually, this paper shows that the LLMs create a lot more slop and degradation of the code after the first prompt.”
“fetch this article for slopcodebench and help me run an eval on a subset of problems with opus 5 SlopCodeBench”
看全部 7 条 →
“Edit: FWIW the paper the post quoted has repositories as slop baseline SlopCodeBench”
- SlopCodeBench: Benchmarking How Coding Agents Degrade over Long-Horizon Tasks ↗FiberBundle · 2026-03-27 · 1 分
- SlopCodeBench: Benchmarking How Coding Agents Degrade over Long-Horizon Task ↗mohsen1 · 2026-04-07 · 1 分
- SlopCodeBench: Benchmarking How Coding Agents Degrade over Long, Iterative Tasks ↗softwaredoug · 2026-07-05 · 1 分