按年被提到
20232023 · 3 次2026
AI 安全、伦理与社会影响›通用智能与意识·arXiv:2302.08399 · cs.AI

Large Language Models Fail on Trivial Alterations to Theory-of-Mind Tasks

Tomer Ullman · 2023
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 3 人首次 2023-06-23 · 最近 2023-10-16
3不同的人
2活跃月份
3讨论串
211被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Intuitive psychology is a pillar of common-sense reasoning. The replication of this reasoning in machine intelligence is an important stepping-stone on the way to human-like artificial intelligence. Several recent tasks and benchmarks for examining this reasoning in Large-Large Models have focused in particular on belief attribution in Theory-of-Mind tasks. These tasks have shown both successes and failures. We consider in particular a recent purported success case, and show that small variations that maintain the principles of ToM turn the results on their head. We argue that in general, the zero-hypothesis for model evaluation in intuitive psychology should be skeptical, and that outlying failure cases should outweigh average success rates. We also consider what possible future successes on Theory-of-Mind tasks by more powerful LLMs would mean for ToM tasks with people.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论mjburgess · 2023-06

“Xp just have to be chosen such that the distribution Xc,Xp is sufficiently small in the training data -- but not that the tokens of Xp are themselves rare. So that an agent competent with tokens in X, who can construct repr of S, could do so with Xp. Consider a reference in the paper above, https://arxiv.org/pdf/2302.08399.pdf Xc = > Here is a bag fill…”

From Word Models to World Models原评论 ↗
评论rvz · 2023-10

“> The implications are vast. We may be able to translate between languages that have never had a “Rosetta Stone”. Any animals that have a true language could have it decoded. And while an LLM that’s gotten an 8 year old’s understanding of balancing assorted items isn’t that useful, an LLM that’s got a baby whale’s grasp on whale language would be revoluti…”

GPT-4 Understands原评论 ↗
被提交为 story · 1 次