按年被提到
20262026 · 4 次2026
算法与数据结构›经典算法与数据结构·arXiv:2604.18555 · cs.LG

A Note on TurboQuant and the Earlier DRIVE/EDEN Line of Work

Ran Ben-Basat, Yaniv Ben-Itzhak, Gal Mendelson, Michael Mitzenmacher, Amit Portnoy, Shay Vargaftik · 2026
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2026-04-27 · 最近 2026-05-02
2不同的人
2活跃月份
3讨论串
0被引(OpenAlex)
arXiv 摘要(候选层不写导读)

This note clarifies the relationship between the recent TurboQuant work and the earlier DRIVE (NeurIPS 2021) and EDEN (ICML 2022) schemes. DRIVE is a 1-bit quantizer that EDEN extended to any $b>0$ bits per coordinate; we refer to them collectively as EDEN. First, TurboQuant$_{\text{mse}}$ is a special case of EDEN obtained by fixing EDEN's scalar scale parameter to $S=1$. EDEN supports both biased and unbiased quantization, each optimized by a different $S$ (chosen via methods described in the EDEN works). The fixed choice $S=1$ used by TurboQuant is generally suboptimal, although the optimal $S$ for biased EDEN converges to $1$ as the dimension grows; accordingly TurboQuant$_{\text{mse}}$ approaches EDEN's behavior for large $d$. Second, TurboQuant$_{\text{prod}}$ combines a biased $(b-1)$-bit EDEN step with an unbiased 1-bit QJL quantization of the residual. It is suboptimal in three ways: (1) its $(b-1)$-bit step uses the suboptimal $S=1$; (2) its 1-bit unbiased residual quantization has worse MSE than (unbiased) 1-bit EDEN; (3) chaining a biased $(b-1)$-bit step with a 1-bit unbiased residual step is inferior to unbiasedly quantizing the input directly with $b$-bit EDEN. Third, some of the analysis in the TurboQuant work mirrors that of the EDEN works: both exploit the connection between random rotations and the shifted Beta distribution, use the Lloyd-Max algorithm, and note that Randomized Hadamard Transforms can replace uniform random rotations. Experiments support these claims: biased EDEN (with optimized $S$) is more accurate than TurboQuant$_{\text{mse}}$, and unbiased EDEN is markedly more accurate than TurboQuant$_{\text{prod}}$, often by more than a bit (e.g., 2-bit EDEN beats 3-bit TurboQuant$_{\text{prod}}$). We also repeat all accuracy experiments from the TurboQuant paper, showing that EDEN outperforms it in every setup we have tried.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论amitport · 2026-04

“TurboQuant IS restricted EDEN quantization (NeurIPS 21, ICML 22). Missing the adaptive scale derivation, which makes TurboQuant variant considerbly LESS optimal than the 22 work. We show this throughfully in a new note in https://arxiv.org/abs/2604.18555 We were the first to introduce post-rotation distribution aware quantization in 21, which was LATER i…”

TurboQuant: A First-Principles Walkthrough原评论 ↗
评论KnuthIsGod · 2026-04

“https://arxiv.org/abs/2604.18555 "This note clarifies the relationship between the recent TurboQuant work and the earlier DRIVE (NeurIPS 2021) and EDEN (ICML 2022) schemes. DRIVE is a 1-bit quantizer that EDEN extended to any bits per coordinate; we refer to them collectively as EDEN. First, TurboQuant is a special case of EDEN obtained by fixing EDEN's …”

TurboQuant: A First-Principles Walkthrough原评论 ↗
评论amitport · 2026-05

“Hi, I’m the author of the post above. tl;dr TurboQuant is a recent paper from Google and NYU that has gained massive traction in mainstream media and the AI community. As implementations of TurboQuant are integrated into various popular projects, it is important to note its relation to EDEN quantization. TurboQuant is essentially a partial implementati…”

被提交为 story · 1 次