“Except that the "content" that reaches the top is always about AI / LLMs and nothing else and it is "all the time". Any opportunity to comment, he will link back to his own blog. He even reposted the same link (which is about AI) with one of his posts when the upvotes fell off and until the second one reached the top, with the intention of promoting his …”
mHC: Manifold-Constrained Hyper-Connections
Recently, studies exemplified by Hyper-Connections (HC) have extended the ubiquitous residual connection paradigm established over the past decade by expanding the residual stream width and diversifying connectivity patterns. While yielding substantial performance gains, this diversification fundamentally compromises the identity mapping property intrinsic to the residual connection, which causes severe training instability and restricted scalability, and additionally incurs notable memory access overhead. To address these challenges, we propose Manifold-Constrained Hyper-Connections (mHC), a general framework that projects the residual connection space of HC onto a specific manifold to restore the identity mapping property, while incorporating rigorous infrastructure optimization to ensure efficiency. Empirical experiments demonstrate that mHC is effective for training at scale, offering tangible performance improvements and superior scalability. We anticipate that mHC, as a flexible and practical extension of HC, will contribute to a deeper understanding of topological architecture design and suggest promising directions for the evolution of foundational models.
候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“https://arxiv.org/abs/2512.24880 was published less than two weeks ago, which should explain why it's not more common yet. And it's not that amazing either. It's a slight quality improvement for a slight increase in cost. It's not even clear to me whether it pays for itself.”
“Was expecting that the release would be this month [0], since everyone forgot about it and not reading the papers they were releasing and 7 days later here we have it. One of the key points of this model to look at is the optimization that DeepSeek made with the residual design of the neural network architecture of the LLM, which is manifold-constrained …”
- MHC: Manifold-Constrained Hyper-Connections ↗ipnon · 2026-01-01 · 1 分
- MHC: Manifold-Constrained Hyper-Connections ↗tamnd · 2026-01-01 · 2 分
- DeepSeek MHC: Manifold-Constrained Hyper-Connections ↗marc__1 · 2026-01-02 · 1 分
- MHC: Manifold-Constrained Hyper-Connections ↗sonabinu · 2026-01-02 · 1 分