“Table 1 in your link is a good way to compare models.”
QLoRA: Efficient Finetuning of Quantized LLMs
4 比特量化加 LoRA,单张 48GB 显卡微调 65B 模型
QLoRA 让梯度反向传播穿过冻结的 4 比特量化预训练模型,只训练低秩适配器(LoRA),从而在单张 48GB GPU 上微调 65B 参数模型,同时保持 16 比特全量微调的任务表现。它引入了 4 比特 NormalFloat(NF4)数据类型、对量化常数再量化的双重量化,以及分页优化器来应对显存峰值。作者用它微调了 1000 多个模型,并训练出 Guanaco 模型系列,同时分析了聊天机器人评测的可靠性。
HN 上的人把它当作在消费级显卡上微调大模型的标准来源,常引用其 Guanaco 结果和 LoRA 作用于哪些层的对比。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“also the original QLoRA paper from Dettmers QLoRA has so many more important morsels.”
“I'd be really curious what the authors of the recent (3 days ago) paper on QLora would think of this article?”
“But, this paper[0] released a week ago claims " 99.3% of the performance level of ChatGPT while only requiring 24 hours of finetuning on a single GPU" (QLORA).”
“A QLoRA finetune of LLaMA-65B by Tim Dettmers from the paper here: QLoRA”
“4. QLoRA: Efficient Finetuning of Quantized LLMs [showing that LLMs can now be fine-tuned quickly on consumer-grade GPUs]”
看全部 9 条 →
“[Research has shown] that low-rank fine-tuning performance stays almost on-par with full fine-tuning.”
“I know QLoRA's paper QLoRA has an approximate calculation of the weights size, but not the gradient updates”
“I think the QLoRA paper QLoRA paper also showed LoRA on all MLP + Attention layers > all MLP layers > just Attention layers.”
- QLoRA: Efficient Finetuning of Quantized LLMs ↗MacsHeadroom · 2023-05-24 · 2 分
- QLoRA: Efficient Finetuning of Quantized LLMs ↗Garcia98 · 2023-05-24 · 1 分