按年被提到
20232023 · 19 次2026
大模型与语言模型›大模型评测与真实性·arXiv:2305.14975 · cs.CL

Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback

直接问就行:从经人类反馈微调的语言模型中获取校准的置信度
Katherine Tian, Eric Mitchell, Allan Zhou, Archit Sharma, Rafael Rafailov, Huaxiu Yao, Chelsea Finn, Christopher D. Manning · 2023

RLHF 模型口头给出的置信度,比其 token 概率校准得更好

开始阅读 ↗PDFalphaXiv 讨论
被提及 · 4 人首次 2023-09-27 · 最近 2026-01-19发表于 Conference on Empirical Methods in Natural Language Processing (EMNLP)
4不同的人
7活跃月份
19讨论串
124被引(OpenAlex)

论文指出,预训练语言模型的条件概率校准得很好,但最常用的经 RLHF 微调的模型,其条件概率被认为校准很差。作者广泛评测了从 RLHF 模型中提取置信度的方法,发现对 ChatGPT、GPT-4 和 Claude,在 TriviaQA、SciQ 和 TruthfulQA 上,作为输出 token 给出的口头置信度通常比模型的条件概率校准得更好,常使期望校准误差相对降低 50%。

HN 为什么提它

评论者在同一份清单里反复把它列为证据,说明模型能给出校准的置信度、知道自己何时在幻觉,后来也用来说明 LLM 内部知道的比说出来的多。

实证研究难度 3 / 5约 40 分钟有时效性
读之前最好知道
1模型校准与期望校准误差(ECE)
2RLHF 微调
3大语言模型提示方法
4问答基准(TriviaQA、TruthfulQA)

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
引用og_kalu · 2023-09

“Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - Just Ask for Calibration”

贴出这篇论文,讲如何从经人类反馈微调的模型中获取校准的置信度
The Internal State of a Large Language Model Knows When Its Lying原评论 ↗
引用og_kalu · 2023-10

“Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - [Just Ask for Calibration)”

引作证据:模型知道自己在幻觉
LLMs Confabulate Not Hallucinate原评论 ↗
引用og_kalu · 2023-10

“Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - [Just Ask for Calibration...”

引作证据:模型知道自己在幻觉
LLMs Confabulate Not Hallucinate原评论 ↗
引用og_kalu · 2023-10

“The prompts here demonstrably help”

指出这里的提示词确实能让模型承认不确定
LLMs Confabulate Not Hallucinate原评论 ↗
引用og_kalu · 2023-10

“Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - Just Ask for Calibration”

引作证据:模型知道自己在幻觉
How Will AI Learn Next?原评论 ↗
引用og_kalu · 2023-10

“Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - Just Ask for Calibration”

引作证据:模型知道自己在幻觉
LLMs can't self-correct in reasoning tasks, DeepMind study finds原评论 ↗
看全部 20 条 →
引用og_kalu · 2023-10

“Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - Just Ask for Calibration”

贴出从 RLHF 微调模型获取校准置信度的方法
LLMs can't self-correct in reasoning tasks, DeepMind study finds原评论 ↗
引用og_kalu · 2023-10

“Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - Just Ask for Calibration”

贴出从 RLHF 微调模型获取校准置信度的方法
ChatGPT and other AI tools could disrupt scientific publishing原评论 ↗
引用og_kalu · 2023-10

“Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - Just Ask for Calibration”

贴出从 RLHF 微调模型获取校准置信度的方法
The Geometry of Truth: Emergent Linear Structure in How LLMs Represent Truth原评论 ↗
引用og_kalu · 2023-10

“Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - Just Ask for Calibration”

贴出从 RLHF 微调模型获取校准置信度的方法
"Hallucinating" AIs Sound Creative, but Let's Not Celebrate Being Wrong原评论 ↗
引用og_kalu · 2023-10

“Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - Just Ask for Calibration”

贴出从 RLHF 微调模型获取校准置信度的方法
The Geometry of Truth: Do LLM's Know True and False原评论 ↗
引用og_kalu · 2023-11

“Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - Just Ask for Calibration”

列入证据清单:从 RLHF 模型获取校准置信度的方法
Ask HN: What is your standard for judging when AGI exists?原评论 ↗
引用og_kalu · 2023-11

“Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - Just Ask for Calibration”

列入证据清单:从 RLHF 模型获取校准置信度的方法
Ask HN: What is your standard for judging when AGI exists?原评论 ↗
引用og_kalu · 2023-11

“Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - Just Ask for Calibration”

列入证据清单:从 RLHF 模型获取校准置信度的方法
Rats have an imagination, new research suggests原评论 ↗
引用og_kalu · 2023-11

“Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - Just Ask for Calibration”

列入证据清单:从 RLHF 模型获取校准置信度的方法
Role play with large language models – Nature原评论 ↗
引用og_kalu · 2023-11

“Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - Just Ask for Calibration”

列入证据清单:从 RLHF 模型获取校准置信度的方法
Language models and linguistic theories beyond words原评论 ↗
引用og_kalu · 2023-11

“Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - Just Ask for Calibration”

列入证据清单:从 RLHF 模型获取校准置信度的方法
I disagree with Geoff Hinton regarding "glorified autocomplete"原评论 ↗
引用cosmojg · 2023-12

“the performance of the model has degraded significantly[2]”

引用它谈 RLHF 微调后模型表现明显下降
Anthropic Claude performance depends heavily on prompt style原评论 ↗
引用famouswaffles · 2024-09

“Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - Just Ask for Calibration”

列作证据:LLM 对不确定性的了解比它说出来的多
AI is growing faster than companies can secure it, warn industry leaders原评论 ↗
引用dcx · 2026-01

“To share some intuition on why one might believe this is occurring: there are a bunch of tasks implicit in the pre-training corpus that encourage the model to learn this capability.”

引出更早的相关工作,讲如何从调优后的模型取得校准的置信度
Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversity原评论 ↗