按年被提到
20252025 · 6 次2026
大模型与语言模型›模型表达力与计算理论·arXiv:2510.15511 · cs.LG

Language Models are Injective and Hence Invertible

Giorgos Nikolaou, Tommaso Mencattini, Donato Crisostomi, Andrea Santilli, Yannis Panagakis, Emanuele Rodolà · 2025
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 7 人首次 2025-10-24 · 最近 2026-04-30
7不同的人
2活跃月份
7讨论串
7被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Transformer components such as non-linear activations and normalization are inherently non-injective, suggesting that different inputs could map to the same output and prevent exact recovery of the input from a model's representations. In this paper, we challenge this view. First, we prove mathematically that transformer language models mapping discrete input sequences to their corresponding sequence of continuous representations are injective and therefore lossless, a property established at initialization and preserved during training. Second, we confirm this result empirically through billions of collision tests on six state-of-the-art language models, and observe no collisions. Third, we operationalize injectivity: we introduce SipIt, the first algorithm that provably and efficiently reconstructs the exact input text from hidden activations, establishing linear-time guarantees and demonstrating exact invertibility in practice. Overall, our work establishes injectivity as a fundamental and exploitable property of language models, with direct implications for transparency, interpretability, and safe deployment.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论Joel_Mckay · 2026-04

“> LLM bot poisoning discourse is against YC site usage policy. Sock-puppet accounts may be banned for AstroTurf or slop. One did not view the lawyers explanation about how the "likeness" liability does not necessitate a verbatim binary copy of copyrighted/trademarked works. The famous-persons criteria was removed in the US due to users posting deep-fake…”

Claude for Creative Work原评论 ↗
被提交为 story · 6 次