“Human experts can reliably detect some kinds of long-form, AI-generated text using exactly the same sorts of cues”
People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated text
常用 LLM 写作的标注者检测 AI 文本,胜过多数检测器
作者雇用标注者阅读 300 篇英文非虚构文章,判断其为人写还是 AI 生成(GPT-4o、Claude、o1),并写出段落式理由。结果显示经常用 LLM 写作的标注者无需专门训练或反馈就很擅长检测,五位这类“专家”的多数票只误判了 300 篇中的 1 篇,明显优于多数商用和开源检测器,即使面对改写和“人性化”等规避手段也如此。定性分析显示他们既依赖具体的词汇线索(“AI 词汇”),也能察觉正式程度、原创性和清晰度等更复杂的现象。数据集与代码已发布。
评论者在争论人类和检测工具(如 Pangram)能否可靠识别 AI 文本时,把它当作独立研究证据:常用 LLM 写作的人准确率很高、误判极少。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“However there is evidence that writers who have experience using LLMs are highly accurate at detecting AI generated text.”
“There are recent papers from academics independently bench-marking and studying detectors e.g. People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated text”
“Not perfect, but I think as good evidence as any:”
“shows 98% for example.”
“Said Pangram report literally citing the single evidence of em-dashes...”
看全部 12 条 →
“It's also confirmed in third-party studies like this People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated text”
“It has extremely low false positive rate in various tests (e. g. People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated text),”
“On the humans (vs. pangram) ability to reliably spot LLM generated content, there was an independent study: People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated text”
“This has been pretty well studied. Pangram is about as good at an expert human with a 98% detection rate and <2% false positive, for flagging AI generated text.”
“There are independent evaluations [1, 2] showing it works,”
“People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated text - People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated text”
- People who use ChatGPT for writing are robust detectors of AI-generated text ↗eatonphil · 2025-01-29 · 2 分
- People who use ChatGPT for writing are accurate detectors of AI-generated text ↗speckx · 2025-05-22 · 1 分
- People who frequently use ChatGPT for writing tasks can detect AI-generated text ↗gaws · 2025-07-17 · 12 分
- Frequent users of ChatGPT are robust detectors of AI text ↗wrp · 2025-07-19 · 1 分
- Frequent ChatGPT users are accurate detectors of AI-generated text (2025) ↗croemer · 2026-04-07 · 1 分
- People who use ChatGPT for writing are accurate detectors of AI text (2025) ↗lexandstuff · 2026-05-19 · 2 分