推荐PaulHoule · 2024-09
“boy do the people in that last paper get a result way better than has been reported so far.”
称这篇论文的 LLM 压缩结果比此前报道的好得多
用 LLaMA-7B 预测下一词,估计英文熵并做无损压缩
论文以 LLaMA-7B 为下一 token 预测器,给出英语熵渐近上界的新估计,数值明显低于此前已有估计。由此得到的副产品是一种无损文本压缩算法:把 LLM 的预测与无损压缩方案结合。有限实验的初步结果显示,其表现优于 BSC、ZPAQ、paq8h 等先进文本压缩方案。
HN 上的人在讨论文本压缩时提它,说明用 LLM 的下一词概率驱动算术编码能得到远好于传统压缩器的结果。
“boy do the people in that last paper get a result way better than has been reported so far.”
“See eg LLMZip and [LINK] or [LINK]”
“You can get much better than that by taking a well-known LLM model and encoding a series of offsets from the most likely sequence of tokens”