“By the way, did you know that this 'big blob of inanimate knowledge' also knows what it doesn't know and can give well-calibrated predictions of accuracy Teaching Models to Express Their Uncertainty in Words ?”
Teaching Models to Express Their Uncertainty in Words
GPT-3 可用文字说出自己答案的置信度,且校准良好
论文表明 GPT-3 可以学会用自然语言(如“90% 置信度”“高置信度”)表达对自己答案的把握,而不依赖 logits,这些置信度与实际正确率大体校准。作者提出 CalibratedMath 任务集,对比了“口头概率”与从 logits 提取的不确定性,两者在分布偏移下都能保持一定校准。结果还显示这种能力依赖预训练中已形成的、与认知不确定性相关的潜在表示。
讨论 LLM 是否知道自己在幻觉、能否分辨真假时,评论者反复把它作为模型能表达校准后不确定性的证据。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“Teaching Models to Express Their Uncertainty in Words -”
“Teaching Models to Express Their Uncertainty in Words - [Teaching Models to Express Their Uncertainty in Words)”
“Teaching Models to Express Their Uncertainty in Words - [Teaching Models to Express Their Uncertainty in Words...”
“Teaching Models to Express Their Uncertainty in Words - Teaching Models to Express Their Uncertainty in Words”
“Teaching Models to Express Their Uncertainty in Words - Teaching Models to Express Their Uncertainty in Words”
看全部 20 条 →
“Teaching Models to Express Their Uncertainty in Words - Teaching Models to Express Their Uncertainty in Words”
“Teaching Models to Express Their Uncertainty in Words - Teaching Models to Express Their Uncertainty in Words”
“Teaching Models to Express Their Uncertainty in Words - Teaching Models to Express Their Uncertainty in Words”
“Teaching Models to Express Their Uncertainty in Words - Teaching Models to Express Their Uncertainty in Words”
“Teaching Models to Express Their Uncertainty in Words - Teaching Models to Express Their Uncertainty in Words”
“Teaching Models to Express Their Uncertainty in Words - Teaching Models to Express Their Uncertainty in Words”
“Teaching Models to Express Their Uncertainty in Words - Teaching Models to Express Their Uncertainty in Words”
“Teaching Models to Express Their Uncertainty in Words - Teaching Models to Express Their Uncertainty in Words”
“Teaching Models to Express Their Uncertainty in Words - Teaching Models to Express Their Uncertainty in Words”
“Teaching Models to Express Their Uncertainty in Words - Teaching Models to Express Their Uncertainty in Words”
“Teaching Models to Express Their Uncertainty in Words - Teaching Models to Express Their Uncertainty in Words”
“Teaching Models to Express Their Uncertainty in Words - Teaching Models to Express Their Uncertainty in Words”
“Teaching Models to Express Their Uncertainty in Words - Teaching Models to Express Their Uncertainty in Words”
“Teaching Models to Express Their Uncertainty in Words - Teaching Models to Express Their Uncertainty in Words”