引用godelski · 2025-08
“We can see them become very good predictors yet the world that they model for is significantly different from the one we live in.”
引它说明模型预测很准,但其隐含的世界与真实世界差异很大
借 Myhill-Nerode 定理评估 LLM 隐含世界模型是否连贯
论文把「LLM 是否隐式学到世界模型」形式化为:底层现实由确定性有限自动机支配的情形,并受语言理论中的 Myhill-Nerode 定理启发提出新的世界模型恢复评估指标。在博弈、逻辑谜题和导航三个领域中,被测生成模型在现有诊断上表现良好,但新指标显示其世界模型远不如表面连贯,这种不连贯会让模型在相关但略有不同的任务上失败。
评论者用它反驳或检验「LLM 已经建立了世界模型」的说法:模型能成为很好的预测器,但其隐含的世界与真实世界差别很大。
“We can see them become very good predictors yet the world that they model for is significantly different from the one we live in.”
“There's actually no strong evidence that LLMs, or any AI system, is actually building a world model.”
“Vafa also has some good examples[1,2].”
“[1] Evaluating the World Model Implicit in a Generative Model”