“Yeah that's odd. It seems like you'd want an n-1 dimensional grid on the surface of the unit sphere rather than an n dimensional grid within which the sphere resides. Looking at the paper (https://arxiv.org/abs/2504.19874) they cite earlier work that does exactly that. They object that grid projection and binary search perform exceptionally poorly on the…”
TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate
Vector quantization, a problem rooted in Shannon's source coding theory, aims to quantize high-dimensional Euclidean vectors while minimizing distortion in their geometric structure. We propose TurboQuant to address both mean-squared error (MSE) and inner product distortion, overcoming limitations of existing methods that fail to achieve optimal distortion rates. Our data-oblivious algorithms, suitable for online applications, achieve near-optimal distortion rates (within a small constant factor) across all bit-widths and dimensions. TurboQuant achieves this by randomly rotating input vectors, inducing a concentrated Beta distribution on coordinates, and leveraging the near-independence property of distinct coordinates in high dimensions to simply apply optimal scalar quantizers per each coordinate. Recognizing that MSE-optimal quantizers introduce bias in inner product estimation, we propose a two-stage approach: applying an MSE quantizer followed by a 1-bit Quantized JL (QJL) transform on the residual, resulting in an unbiased inner product quantizer. We also provide a formal proof of the information-theoretic lower bounds on best achievable distortion rate by any vector quantizer, demonstrating that TurboQuant closely matches these bounds, differing only by a small constant ($\approx 2.7$) factor. Experimental results validate our theoretical findings, showing that for KV cache quantization, we achieve absolute quality neutrality with 3.5 bits per channel and marginal quality degradation with 2.5 bits per channel. Furthermore, in nearest neighbor search tasks, our method outperforms existing product quantization techniques in recall while reducing indexing time to virtually zero.
候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“The blog is new but the paper was submitted almost one year ago: https://arxiv.org/abs/2504.19874. Anyone has ideas if this is already implemented in many models (at least Gemini, I guess)? If that's the case, can I expect cheaper RAM for my computer:D”
“The actual paper from April 2025 TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate https://arxiv.org/abs/2504.19874”
“It looks nice! Fair enough about QJL - it seems to be nothing more than an unbiasing measure anyway. I'm not sure if it's my own misunderstanding or if the paper [0] has something of an error. Section 3.1 starts out to the effect "let x be on the unit hypersphere" (but I'm fairly certain it's actually not). Neither algorithm 1 nor algorithm 2 show a norm…”
“> Google announced Turboquant[1] today Most commentators seem to have missed that the TurboQuant paper was posted in April 2025: https://arxiv.org/abs/2504.19874 So it's been public knowledge for almost year, and internal Google knowledge for longer. Did Google reduce its capital expenditure plans over the past year? Let's ask Google: Google is expect…”
“I built a Python implementation of Google's TurboQuant paper (ICLR 2026) for vector search. The key thing that makes this different from PQ and other quantization methods: it's fully data-oblivious. The codebook is derived from math (not trained on your data), so you can add vectors online without ever rebuilding the index. Each vector encodes independent…”
看全部 10 条 →
“The PolarQuant paper: https://arxiv.org/abs/2502.02617 The TurboQuant paper: https://arxiv.org/abs/2504.19874 These elaborate ideas are not adequately represented in the article.”
“It especially doesn't make sense considering that TurboQuant has been public on arXiv for almost a year: https://arxiv.org/abs/2504.19874 So it predates the late-2025 RAM price surge! https://pcpartpicker.com/trends/price/memory/ I think that either investors were extremely skittish that the stocks might crash and jumped at the first sign of trouble (cre…”
“BTW, a number of corrections. The TurboQuant paper was submitted to Arxiv back in April 2025: https://arxiv.org/abs/2504.19874 Current "TurboQuant" implementations are about 3.8X-4.9X on compression (w/ the higher end taking some significant hits of GSM8K performance) and with about 80-100% baseline speed (no improvement, regression): https://github.com/…”
“> To be fair, TurboQuant is also a purely theoretical paper. I just wanted to get this out and share. TurboQuant is not a purely theoretical paper. Section 4 "Experiments" (page 15) [0] has a bunch of figure based on actual GPU computations. [0]: https://arxiv.org/abs/2504.19874”
- Google's extreme AI compression paper was on arXiv since April 2025 ↗fadijob · 2026-03-25 · 1 分