按年被提到
20222023 · 2 次2026
大模型与语言模型›词向量与预训练模型·arXiv:2212.09410 · cs.CL

Less is More: Parameter-Free Text Classification with Gzip

Zhiying Jiang, Matthew Y. R. Yang, Mikhail Tsirlin, Raphael Tang, Jimmy Lin · 2022
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 3 人首次 2023-07-15 · 最近 2026-09-22
3不同的人
2活跃月份
3讨论串
9被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Deep neural networks (DNNs) are often used for text classification tasks as they usually achieve high levels of accuracy. However, DNNs can be computationally intensive with billions of parameters and large amounts of labeled data, which can make them expensive to use, to optimize and to transfer to out-of-distribution (OOD) cases in practice. In this paper, we propose a non-parametric alternative to DNNs that's easy, light-weight and universal in text classification: a combination of a simple compressor like gzip with a $k$-nearest-neighbor classifier. Without any training, pre-training or fine-tuning, our method achieves results that are competitive with non-pretrained deep learning methods on six in-distributed datasets. It even outperforms BERT on all five OOD datasets, including four low-resource languages. Our method also performs particularly well in few-shot settings where labeled data are too scarce for DNNs to achieve a satisfying accuracy.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论bravura · 2023-07

“Compression based ML approaches and hacks are fascinating. My colleague recently sent me this: https://arxiv.org/abs/2212.09410 "Less is More: Parameter-Free Text Classification with Gzip "...We propose a non-parametric alternative to DNNs that's easy, light-weight and universal in text classification: a combination of a simple compressor like gzip wit…”

Ziplm: Gzip-Backed Language Model原评论 ↗
评论AlbertoGP · 2023-07

“This is interesting, but why don’t they link to the paper? Or at least give its title? The paper is «“Low-Resource” Text Classification: A Parameter-Free Classification Method with Compressors»: ACL Anthology, has PDF link: https://aclanthology.org/2023.findings-acl.426/ Arxiv, has PDF link: https://arxiv.org/abs/2212.09410 arXiv Vanity, HTML render o…”

14-line code can be used to 'understand' similarities between sentences原评论 ↗
评论teh64 · 2026-09

“There is an older paper that also looks at gzip for ML: https://arxiv.org/abs/2212.09410 Video from tsoding where he implements the algorithm: https://www.youtube.com/watch?v=9n39SbRPXKQ”

Can gzip be a language model?原评论 ↗