按年被提到
20242024 · 2 次2026
AI 安全、伦理与社会影响›AI 的经济与社会影响·arXiv:2404.03502 · cs.AI

AI and the Problem of Knowledge Collapse

Andrew J. Peterson · 2024
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2024-04-05 · 最近 2026-02-09
2不同的人
4活跃月份
4讨论串
63被引(OpenAlex)
arXiv 摘要(候选层不写导读)

While artificial intelligence has the potential to process vast amounts of data, generate new insights, and unlock greater productivity, its widespread adoption may entail unforeseen consequences. We identify conditions under which AI, by reducing the cost of access to certain modes of knowledge, can paradoxically harm public understanding. While large language models are trained on vast amounts of diverse data, they naturally generate output towards the 'center' of the distribution. This is generally useful, but widespread reliance on recursive AI systems could lead to a process we define as "knowledge collapse", and argue this could harm innovation and the richness of human understanding and culture. However, unlike AI models that cannot choose what data they are trained on, humans may strategically seek out diverse forms of knowledge if they perceive them to be worthwhile. To investigate this, we provide a simple model in which a community of learners or innovators choose to use traditional methods or to rely on a discounted AI-assisted process and identify conditions under which knowledge collapse occurs. In our default model, a 20% discount on AI-generated content generates public beliefs 2.3 times further from the truth than when there is no discount. An empirical approach to measuring the distribution of LLM outputs is provided in theoretical terms and illustrated through a specific example comparing the diversity of outputs across different models and prompting styles. Finally, based on the results, we consider further research directions to counteract such outcomes.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论thesz · 2024-08

“> Its main purpose is to produce synthetic data for Orion, their next big LLM https://arxiv.org/abs/2404.03502 "This is generally useful, but widespread reliance on recursive AI systems could lead to a process we define as "knowledge collapse", and argue this could harm innovation and the richness of human understanding and culture... In our default mode…”

OpenAI shows 'Strawberry' to feds, races to launch it原评论 ↗
评论thesz · 2025-01

“> Your domain is weird/rare, so LLMs are terrible because their training data is very limited. And this is how knowledge collapse [1] shows its' head. [1] https://arxiv.org/abs/2404.03502 I am not a big fan of LLMs so I try them once in a while, asking to "implement blocked clause decomposition in Haskell." They can recite a paper (and several other pap…”

Trae: An AI-powered IDE by ByteDance原评论 ↗
评论thesz · 2026-02

“You are right about "LLM improving themselves" is impossible. In fact, LLM make themselves worse, it is called knowledge collapse [1]. [1] https://arxiv.org/abs/2404.03502”

(AI) Slop Terrifies Me原评论 ↗
被提交为 story · 1 次