“Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - Just Ask for Calibration”
Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback
RLHF 模型口头给出的置信度,比其 token 概率校准得更好
论文指出,预训练语言模型的条件概率校准得很好,但最常用的经 RLHF 微调的模型,其条件概率被认为校准很差。作者广泛评测了从 RLHF 模型中提取置信度的方法,发现对 ChatGPT、GPT-4 和 Claude,在 TriviaQA、SciQ 和 TruthfulQA 上,作为输出 token 给出的口头置信度通常比模型的条件概率校准得更好,常使期望校准误差相对降低 50%。
评论者在同一份清单里反复把它列为证据,说明模型能给出校准的置信度、知道自己何时在幻觉,后来也用来说明 LLM 内部知道的比说出来的多。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - [Just Ask for Calibration)”
“Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - [Just Ask for Calibration...”
“The prompts here demonstrably help”
“Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - Just Ask for Calibration”
“Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - Just Ask for Calibration”
看全部 20 条 →
“Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - Just Ask for Calibration”
“Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - Just Ask for Calibration”
“Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - Just Ask for Calibration”
“Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - Just Ask for Calibration”
“Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - Just Ask for Calibration”
“Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - Just Ask for Calibration”
“Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - Just Ask for Calibration”
“Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - Just Ask for Calibration”
“Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - Just Ask for Calibration”
“Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - Just Ask for Calibration”
“Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - Just Ask for Calibration”
“the performance of the model has degraded significantly[2]”
“Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - Just Ask for Calibration”
“To share some intuition on why one might believe this is occurring: there are a bunch of tasks implicit in the pre-training corpus that encourage the model to learn this capability.”