“Excellent background on knowledge calibration from Anthropic:”
Language Models (Mostly) Know What They Know
大模型能校准地评估自己答案是否正确,并预测自己是否会答对
作者研究语言模型能否评估自己论断的正确性并预测能答对哪些问题。首先表明在合适的格式下,较大模型在多样的选择题和判断题上校准良好;于是让模型先给出答案,再评估该答案正确的概率 P(True),其表现、校准性和缩放趋势都令人鼓舞,参考多个自身样本后效果进一步提升。随后训练模型预测不针对具体答案的 P(IK),即「我知道答案」的概率,模型预测得不错且部分能跨任务泛化,但在新任务上校准较差。
HN 上的人在讨论幻觉时反复把它当作证据,说明大模型大体知道自己的答案是否正确、能分辨真伪,因此有望降低幻觉。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“The linked page from OpenAI clearly show the opposite, read the "Using SimpleQA to measure the calibration of large language models" and the paper linked: Language Models (Mostly) Know What They Know”
“Given that language models seem to mostly know when they're making correct predictions [1]”
“There was a recent work (Language Models (Mostly) Know What They Know, in LLM, not molecular prediction) where models could be shown to”
“It has already been done: Language Models (Mostly) Know What They Know”
“Language Models (Mostly) Know What They Know -”
看全部 20 条 →
“Language Models (Mostly) Know What They Know - [Language Models (Mostly) Know What They Know)”
“Language Models (Mostly) Know What They Know - [Language Models (Mostly) Know What They Know...”
“Language Models (Mostly) Know What They Know - Language Models (Mostly) Know What They Know”
“Language Models (Mostly) Know What They Know - Language Models (Mostly) Know What They Know”
“Language Models (Mostly) Know What They Know - Language Models (Mostly) Know What They Know”
“Language Models (Mostly) Know What They Know - Language Models (Mostly) Know What They Know”
“Language Models (Mostly) Know What They Know - Language Models (Mostly) Know What They Know”
“Language Models (Mostly) Know What They Know - Language Models (Mostly) Know What They Know”
“Language Models (Mostly) Know What They Know - Language Models (Mostly) Know What They Know”
“Language Models (Mostly) Know What They Know - Language Models (Mostly) Know What They Know”
“Language Models (Mostly) Know What They Know - Language Models (Mostly) Know What They Know”
“Language Models (Mostly) Know What They Know - Language Models (Mostly) Know What They Know”
“Language Models (Mostly) Know What They Know - Language Models (Mostly) Know What They Know”
“I believe it's a reference to the paper "Language Models (Mostly) Know What They Know".”