按年被提到
20252026 · 11 次2026
大模型与语言模型›量化、解码与分词·arXiv:2504.19874 · cs.LG

TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate

Amir Zandieh, Majid Daliri, Majid Hadian, Vahab Mirrokni · 2025
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 10 人首次 2026-03-25 · 最近 2026-04-21
10不同的人
2活跃月份
8讨论串
9被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Vector quantization, a problem rooted in Shannon's source coding theory, aims to quantize high-dimensional Euclidean vectors while minimizing distortion in their geometric structure. We propose TurboQuant to address both mean-squared error (MSE) and inner product distortion, overcoming limitations of existing methods that fail to achieve optimal distortion rates. Our data-oblivious algorithms, suitable for online applications, achieve near-optimal distortion rates (within a small constant factor) across all bit-widths and dimensions. TurboQuant achieves this by randomly rotating input vectors, inducing a concentrated Beta distribution on coordinates, and leveraging the near-independence property of distinct coordinates in high dimensions to simply apply optimal scalar quantizers per each coordinate. Recognizing that MSE-optimal quantizers introduce bias in inner product estimation, we propose a two-stage approach: applying an MSE quantizer followed by a 1-bit Quantized JL (QJL) transform on the residual, resulting in an unbiased inner product quantizer. We also provide a formal proof of the information-theoretic lower bounds on best achievable distortion rate by any vector quantizer, demonstrating that TurboQuant closely matches these bounds, differing only by a small constant ($\approx 2.7$) factor. Experimental results validate our theoretical findings, showing that for KV cache quantization, we achieve absolute quality neutrality with 3.5 bits per channel and marginal quality degradation with 2.5 bits per channel. Furthermore, in nearest neighbor search tasks, our method outperforms existing product quantization techniques in recall while reducing indexing time to virtually zero.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论fc417fc802 · 2026-03

“Yeah that's odd. It seems like you'd want an n-1 dimensional grid on the surface of the unit sphere rather than an n dimensional grid within which the sphere resides. Looking at the paper (https://arxiv.org/abs/2504.19874) they cite earlier work that does exactly that. They object that grid projection and binary search perform exceptionally poorly on the…”

TurboQuant: Redefining AI efficiency with extreme compression原评论 ↗
评论wbsun · 2026-03

“The blog is new but the paper was submitted almost one year ago: https://arxiv.org/abs/2504.19874. Anyone has ideas if this is already implemented in many models (at least Gemini, I guess)? If that's the case, can I expect cheaper RAM for my computer:D”

TurboQuant: Redefining AI efficiency with extreme compression原评论 ↗
评论htrp · 2026-03

“The actual paper from April 2025 TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate https://arxiv.org/abs/2504.19874”

TurboQuant: Redefining AI efficiency with extreme compression原评论 ↗
评论fc417fc802 · 2026-03

“It looks nice! Fair enough about QJL - it seems to be nothing more than an unbiasing measure anyway. I'm not sure if it's my own misunderstanding or if the paper [0] has something of an error. Section 3.1 starts out to the effect "let x be on the unit hypersphere" (but I'm fairly certain it's actually not). Neither algorithm 1 nor algorithm 2 show a norm…”

TurboQuant: Redefining AI efficiency with extreme compression原评论 ↗
评论T-A · 2026-03

“> Google announced Turboquant[1] today Most commentators seem to have missed that the TurboQuant paper was posted in April 2025: https://arxiv.org/abs/2504.19874 So it's been public knowledge for almost year, and internal Google knowledge for longer. Did Google reduce its capital expenditure plans over the past year? Let's ask Google: Google is expect…”

Hold on to Your Hardware原评论 ↗
评论justsomeguy1996 · 2026-03

“I built a Python implementation of Google's TurboQuant paper (ICLR 2026) for vector search. The key thing that makes this different from PQ and other quantization methods: it's fully data-oblivious. The codebook is derived from math (not trained on your data), so you can add vectors online without ever rebuilding the index. Each vector encodes independent…”

Show HN: TurboQuant for vector search – 2-4 bit compression原评论 ↗
看全部 10 条 →
评论amai · 2026-03

“The PolarQuant paper: https://arxiv.org/abs/2502.02617 The TurboQuant paper: https://arxiv.org/abs/2504.19874 These elaborate ideas are not adequately represented in the article.”

Google's TurboQuant AI-compression algorithm can reduce LLM memory usage by 6x原评论 ↗
评论yorwba · 2026-03

“It especially doesn't make sense considering that TurboQuant has been public on arXiv for almost a year: https://arxiv.org/abs/2504.19874 So it predates the late-2025 RAM price surge! https://pcpartpicker.com/trends/price/memory/ I think that either investors were extremely skittish that the stocks might crash and jumped at the first sign of trouble (cre…”

What if AI doesn't need more RAM but better math?原评论 ↗
评论lhl · 2026-04

“BTW, a number of corrections. The TurboQuant paper was submitted to Arxiv back in April 2025: https://arxiv.org/abs/2504.19874 Current "TurboQuant" implementations are about 3.8X-4.9X on compression (w/ the higher end taking some significant hits of GSM8K performance) and with about 80-100% baseline speed (no improvement, regression): https://github.com/…”

The RAM shortage could last years原评论 ↗
评论thethirdone · 2026-04

“> To be fair, TurboQuant is also a purely theoretical paper. I just wanted to get this out and share. TurboQuant is not a purely theoretical paper. Section 4 "Experiments" (page 15) [0] has a bunch of figure based on actual GPU computations. [0]: https://arxiv.org/abs/2504.19874”

KV Cache Compression 900000x Beyond TurboQuant and Per-Vector Shannon Limit原评论 ↗
被提交为 story · 1 次