按年被提到
20232023 · 2 次2026
AI 安全、伦理与社会影响›大模型与人类行为·arXiv:2308.10800 · cs.HC

Fact-checking information from large language models can decrease headline discernment

Matthew R. DeVerna, Harry Yaojun Yan, Kai-Cheng Yang, Filippo Menczer · 2023
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2023-08-22 · 最近 2023-09-08
2不同的人
2活跃月份
2讨论串
86被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Fact checking can be an effective strategy against misinformation, but its implementation at scale is impeded by the overwhelming volume of information online. Recent artificial intelligence (AI) language models have shown impressive ability in fact-checking tasks, but how humans interact with fact-checking information provided by these models is unclear. Here, we investigate the impact of fact-checking information generated by a popular large language model (LLM) on belief in, and sharing intent of, political news headlines in a preregistered randomized control experiment. Although the LLM accurately identifies most false headlines (90%), we find that this information does not significantly improve participants' ability to discern headline accuracy or share accurate news. In contrast, viewing human-generated fact checks enhances discernment in both cases. Subsequent analysis reveals that the AI fact-checker is harmful in specific cases: it decreases beliefs in true headlines that it mislabels as false and increases beliefs in false headlines that it is unsure about. On the positive side, AI fact-checking information increases the sharing intent for correctly labeled true headlines. When participants are given the option to view LLM fact checks and choose to do so, they are significantly more likely to share both true and false news but only more likely to believe false headlines. Our findings highlight an important source of potential harm stemming from AI applications and underscore the critical need for policies to prevent or mitigate such unintended consequences.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

没有评论直接链到它;下面是它被提交的讨论串里回复最多的评论(不计入统计)
讨论串评论westurner · 2023-08

“What about systems like pdfgpt and knowledge_gpt; that, in returning citations from the trained texts, might be useful for legal discovery with admissibility? pdfgpt: https://github.com/bhaskatripathi/pdfGPT knowledge_gpt: https://github.com/mmz-001/knowledge_gpt Are LLM tools better or worse than e.g. meilisearch or elasticsearch for searching with sn…”

Artificial intelligence is ineffective and potentially harmful for fact checking原评论 ↗
讨论串评论simonmesmith · 2023-09

“So, some of the findings in relation to the psychology of AI fact-checking they found may be valid and interesting (e.g. how it increases confidence, even in false information when incorrect). But their methodology to me seems quite flawed: 1. They use “ChatGPT” and never mention if it’s 3.5 or 4 (unless I missed it—don’t think so) so I suspect 3.5. 2. …”

Artificial intelligence is ineffective and potentially harmful for fact checking原评论 ↗
被提交为 story · 2 次