推荐Der_Einzige · 2025-05
“our min_p paper was ranked #18 out of 12000 submission at ICLR and got an oral:”
介绍自己的 min_p 论文,在 ICLR 上获口头报告
按最高概率动态截断的 min-p 采样,高温下兼顾质量与多样性
top-p 等常用采样方法难以兼顾质量与多样性,在高温度下尤其容易出现不连贯或重复。论文提出 min-p 采样:以最高概率 token 的概率为比例因子,根据模型置信度动态调整截断阈值。在 GPQA、GSM8K 和 AlpacaEval Creative Writing 上,它在 Mistral 和 Llama 3 等多个模型族、1B 到 123B 的规模上提升质量和多样性,人工评测也偏好它,并已被多个开源框架采用。
HN 上有人(包括作者)提到它是 ICLR oral,并以它说明利用 logprob 做动态截断能显著改进采样;也有评论者指出表 2 中数学任务上 min-p 低于 top-p。
“our min_p paper was ranked #18 out of 12000 submission at ICLR and got an oral:”
“Unless I'm reading Table2 (page7 - pdf version) wrong, on math, min_p is shown to score worse than top_p.”
“You and the OP talk a lot of smack about logprobs but we show that using them in even the simple case of dynamic truncation of your cutoff point (min_p sampler vs static top_p/top_k) leads to extreme performance improvements”