按年被提到
20252026 · 7 次2026
系统、数据库与网络›计算机体系结构与硬件·arXiv:2512.24880 · cs.CL

mHC: Manifold-Constrained Hyper-Connections

Zhenda Xie, Yixuan Wei, Huanqi Cao, Chenggang Zhao, Chengqi Deng, Jiashi Li, Damai Dai, Huazuo Gao, Jiang Chang, Kuai Yu, Liang Zhao, Shangyan Zhou 等 20 人 · 2025
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 6 人首次 2026-01-01 · 最近 2026-04-24
6不同的人
2活跃月份
7讨论串
14被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Recently, studies exemplified by Hyper-Connections (HC) have extended the ubiquitous residual connection paradigm established over the past decade by expanding the residual stream width and diversifying connectivity patterns. While yielding substantial performance gains, this diversification fundamentally compromises the identity mapping property intrinsic to the residual connection, which causes severe training instability and restricted scalability, and additionally incurs notable memory access overhead. To address these challenges, we propose Manifold-Constrained Hyper-Connections (mHC), a general framework that projects the residual connection space of HC onto a specific manifold to restore the identity mapping property, while incorporating rigorous infrastructure optimization to ensure efficiency. Empirical experiments demonstrate that mHC is effective for training at scale, offering tangible performance improvements and superior scalability. We anticipate that mHC, as a flexible and practical extension of HC, will contribute to a deeper understanding of topological architecture design and suggest promising directions for the evolution of foundational models.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论rvz · 2026-01

“Except that the "content" that reaches the top is always about AI / LLMs and nothing else and it is "all the time". Any opportunity to comment, he will link back to his own blog. He even reposted the same link (which is about AI) with one of his posts when the upvotes fell off and until the second one reached the top, with the intention of promoting his …”

2025: The Year in LLMs原评论 ↗
评论yorwba · 2026-01

“https://arxiv.org/abs/2512.24880 was published less than two weeks ago, which should explain why it's not more common yet. And it's not that amazing either. It's a slight quality improvement for a slight increase in cost. It's not even clear to me whether it pays for itself.”

评论rvz · 2026-04

“Was expecting that the release would be this month [0], since everyone forgot about it and not reading the papers they were releasing and 7 days later here we have it. One of the key points of this model to look at is the optimization that DeepSeek made with the residual design of the neural network architecture of the LLM, which is manifold-constrained …”

DeepSeek v4原评论 ↗
被提交为 story · 4 次