按年被提到
20222023 · 9 次2026
大模型与语言模型›量化、解码与分词·arXiv:2212.09720 · cs.LG

The case for 4-bit precision: k-bit Inference Scaling Laws

支持 4 比特精度的理由:k 比特推理缩放定律
Tim Dettmers, Luke Zettlemoyer · 2022

35,000 多组实验表明:模型总比特数固定时,4 比特量化几乎总是最优

开始阅读 ↗PDFalphaXiv 讨论
被推荐 · 10 人首次 2023-01-03 · 最近 2026-06-29
10不同的人
10活跃月份
15讨论串
46被引(OpenAlex)

同样的总比特数下,30B 的 8 比特模型和 60B 的 4 比特模型的零样本精度可能差别很大。作者建立 LLM 零样本性能的推理缩放定律,用 35,000 多次实验在 BLOOM、OPT、NeoX/Pythia 和 GPT-2 系列、19M 到 176B 参数、3 到 8 比特精度下比较各种量化方法。结果显示很难改善比特级缩放的权衡,仅有的改进来自较小的分块大小和量化数据类型(如 Int 对 Float),总体上 4 比特精度几乎普遍对总模型比特数和零样本精度最优。

HN 为什么提它

HN 上的人反复用它说明 4 比特量化是模型大小与精度之间的最佳折中,认为几乎不损失输出质量,并在讨论本地运行大模型和降低推理成本时引用。

实证研究难度 3 / 5约 50 分钟经得起时间
读之前最好知道
1神经网络量化(quantization)
2大语言模型推理与显存占用
3缩放定律(scaling laws)
4零样本评测(zero-shot evaluation)

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
推荐MacsHeadroom · 2023-03

“Read The Case for 4 Bit Precision. The case for 4-bit precision”

推荐阅读它,说明参数量越大,4 比特量化几乎无损
Dalai: Automatically install, run, and play with LLaMA on your computer原评论 ↗
推荐corvec · 2023-03

“Check out the original paper on quantization, which has some benchmarks: [OTHER_PAPER] and this paper, which also has benchmarks and explains how they determined that 4-bit quantization is optimal compared to 3-bit: The case for 4-bit precision”

推荐它,其基准和分析说明 4 比特量化比 3 比特更优
Llama.cpp 30B runs with only 6GB of RAM now原评论 ↗
推荐nok22kon · 2026-06

“paper is a bit old, but matches current empirical recommandation”

推荐它,虽然有点老,但与当前经验建议吻合:4 比特通常最划算
Qwen 3.6 27B is the sweet spot for local development原评论 ↗
引用woodson · 2023-01

“I think quantization (e.g. 4-bit, The case for 4-bit precision) and sparsity (e.g. SparseGPT, [OTHER_PAPER]) will bring down inference cost.”

引它为例,认为量化和稀疏化会降低推理成本
Some Remarks on Large Language Models – Yoav Goldberg原评论 ↗
引用MacsHeadroom · 2023-03

“See also: The case for 4-bit precision, which shows effectively no output quality reduction for these 4bit quantization methods (and considerable speedup) The case for 4-bit precision”

用它说明这些 4 比特量化方法几乎不降低输出质量,还显著提速
Visual ChatGPT原评论 ↗
引用MacsHeadroom · 2023-03

“The case for 4-bit precision - The case for 4-bit precision: k-bit Inference Scaling Laws [Dec, 2022]”

把它列入参考,说明 k 比特推理缩放定律与 4 比特质量相关
LLaMA-7B in Pure C++ with full Apple Silicon support原评论 ↗
看全部 14 条 →
引用MacsHeadroom · 2023-03

“The case for 4-bit precision - The case for 4-bit precision: k-bit Inference Scaling Laws [Dec, 2022]”

列为参考,用缩放定律说明 4 比特量化几乎不损伤性能
Alpaca- An Instruct Tuned Llama 7B. Responses on par with txt-DaVinci-3. Demo up原评论 ↗
引用lhl · 2023-08

“Tim Dettmers did a bunch of research on this last year as well: The case for 4-bit precision”

指出 Dettmers 去年就研究过量化模型与更小未量化模型的对比
How Is LLaMa.cpp Possible?原评论 ↗
引用abhgh · 2023-11

“The tradeoff evidently works favorably till upto 4-bits; I'm basing this off the results reported on the k-bit inference scaling laws paper”

依据它的结果,称权衡到 4 比特为止都划算
QUIK is a method for quantizing LLM post-training weights to 4 bit precision原评论 ↗
引用hnfong · 2024-01

“Generally 4bits+ achieves a good tradeoff, see eg. The case for 4-bit precision”

引它说明 4 比特及以上通常能取得不错的精度权衡
Brave Leo now uses Mixtral 8x7B as default原评论 ↗
引用coder543 · 2024-01

“Various papers have shown that 4-bit quantization is a great balance. One example: The case for 4-bit precision”

举它为例,说明 4 比特量化是很好的平衡点
Meta AI releases Code Llama 70B原评论 ↗
引用hnfong · 2024-04

“(eg. The case for 4-bit precision )”

引它的困惑度结果,说明 4 比特量化一般可以接受
Qwen1.5-110B原评论 ↗
引用maratc · 2024-11

“arguably good results can be achieved even with 4 bit precision [0][1].”

引它说明即使只有 4 比特精度,也可能得到不错的结果
AMD now has more compute on the top 500 than Nvidia原评论 ↗
引用hnfong · 2025-03

“There are papers to that effect though admittedly perhaps specific models might have divergent behavior”

称有论文支持 4 比特量化总体表现良好,但个别模型可能不同
Gemma3 – The current strongest model that fits on a single GPU原评论 ↗
被提交为 story · 1 次