引用gandalfgeek · 2024-06
“some recent research (some of which they cite) has actually quantitatively shown that RAG is much preferable to FT for adding domain-specific knowledge to an LLM:”
引用近期研究(其中一些被引用过)说明为 LLM 添加领域知识,RAG 定量上优于微调
实验显示注入知识时 RAG 稳定优于无监督微调
论文比较向大语言模型注入新知识的两种常见做法:无监督微调与检索增强生成(RAG),并在不同主题的知识密集型任务上评估。结果显示无监督微调有一定提升,但 RAG 始终更好,无论是对训练中见过的已有知识,还是全新的知识。作者还发现 LLM 很难通过无监督微调学到新的事实,而在训练时让它看到同一事实的多种不同表述可能缓解这一问题。
评论者引用它作为定量证据,说明为 LLM 增加领域知识时 RAG 比微调更可取。
“some recent research (some of which they cite) has actually quantitatively shown that RAG is much preferable to FT for adding domain-specific knowledge to an LLM:”