“The LIMA paper, I think, would be a good place to start LIMA”
LIMA: Less Is More for Alignment
仅用 1000 条精选样本微调,就能让大模型很好地对齐
论文用标准监督损失,仅在 1000 条精心挑选的提示与回答上微调 65B 的 LLaMa 模型,不用任何强化学习或人类偏好建模,得到 LIMA。在受控人工评测中,LIMA 的回答在 43% 的情况下与 GPT-4 相当或更受偏好,对 Bard 为 58%,对经人类反馈训练的 DaVinci003 为 65%。作者据此认为大模型的几乎所有知识都来自预训练,只需少量指令微调数据就能教会模型产出高质量回答。
HN 上的人用它说明“数据质量和数量同样重要”:极少量高质量样本就足以对齐基础模型,并把它当作了解微调与对齐的入门起点。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“They're doing it right now: LIMA”
“LIMA: Less Is More for Alignment [showing that a very small number of high quality examples is sufficient to align a base model]”
“secondary supervised learning stages actually can get by with a far smaller set of highly curated prompt-response data, e.g. LIMA.”
“Quality matters just as much as quantity.”
“LIMA: Less Is More for Alignment”
看全部 7 条 →
“Meta has a paper out that argues you only need a data set of 1000[1] to achieve pretty good alignment (fine-tuning) results.”
- LIMA: Less Is More for Alignment ↗lebek · 2023-05-22 · 4 分
- LIMA: Less Is More for Alignment ↗digitcatphd · 2023-05-22 · 1 分