“these papers boast their own methods by poorly implementing a baseline and describing their own methods using lots of mathematical jargon”
SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot
无需重训,一次性把 GPT 类大模型剪到 50% 稀疏几乎不掉精度
论文首次表明 GPT 系列大模型可以一次性(one-shot)剪枝到至少 50% 稀疏度而无需重新训练,精度损失很小,所用方法 SparseGPT 专为超大模型设计。它能在不到 4.5 小时内处理 OPT-175B 和 BLOOM-176B,达到 60% 非结构化稀疏度而困惑度几乎不升;还支持 2:4、4:8 等半结构化模式,并与权重量化兼容。
评论者把它与量化并列,作为降低推理成本的途径:可以剪掉约一半参数而几乎不损失性能;另有一条批评认为这类论文常靠弱基线和数学术语包装,属泛泛之论。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“I think quantization (e.g. 4-bit, [OTHER_PAPER]) and sparsity (e.g. SparseGPT, SparseGPT) will bring down inference cost.”
“Here's a technique SparseGPT which suggests that you can prune 50% of parameters with almost no loss in performance for example: SparseGPT.”
“combined with sparsity pruning (-50% size w/ almost no loss in accuracy) [2]”
“The same group did another paper SparseGPT which shows that in addition to reducing the precision of each parameter, you can also prune out a bunch of parameters entirely.”
“Relevantish: SparseGPT”
- SparseGPT: Language Models Can Be Accurately Pruned in One-Shot ↗tosh · 2023-05-03 · 24 分