按年被提到
20252025 · 7 次2026
大模型与语言模型›混合专家与高效训练·arXiv:2505.09343 · cs.DC

Insights into DeepSeek-V3: Scaling Challenges and Reflections on Hardware for AI Architectures

Chenggang Zhao, Chengqi Deng, Chong Ruan, Damai Dai, Huazuo Gao, Jiashi Li, Liyue Zhang, Panpan Huang, Shangyan Zhou, Shirong Ma, Wenfeng Liang, Ying He 等 15 人 · 2025
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 6 人首次 2025-05-15 · 最近 2025-08-09
6不同的人
2活跃月份
6讨论串
60被引(OpenAlex)
arXiv 摘要(候选层不写导读)

The rapid scaling of large language models (LLMs) has unveiled critical limitations in current hardware architectures, including constraints in memory capacity, computational efficiency, and interconnection bandwidth. DeepSeek-V3, trained on 2,048 NVIDIA H800 GPUs, demonstrates how hardware-aware model co-design can effectively address these challenges, enabling cost-efficient training and inference at scale. This paper presents an in-depth analysis of the DeepSeek-V3/R1 model architecture and its AI infrastructure, highlighting key innovations such as Multi-head Latent Attention (MLA) for enhanced memory efficiency, Mixture of Experts (MoE) architectures for optimized computation-communication trade-offs, FP8 mixed-precision training to unlock the full potential of hardware capabilities, and a Multi-Plane Network Topology to minimize cluster-level network overhead. Building on the hardware bottlenecks encountered during DeepSeek-V3's development, we engage in a broader discussion with academic and industry peers on potential future hardware directions, including precise low-precision computation units, scale-up and scale-out convergence, and innovations in low-latency communication fabrics. These insights underscore the critical role of hardware and model co-design in meeting the escalating demands of AI workloads, offering a practical blueprint for innovation in next-generation AI systems.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论qtwhat · 2025-05

“DeepSeek-V3 demonstrates that thoughtful hardware-software co-design can overcome the scaling challenges of large language models. By integrating innovations like Multi-head Latent Attention (MLA), Mixture of Experts (MoE) architectures, FP8 mixed-precision training, and a Multi-Plane Network Topology, DeepSeek-V3 achieves cost-effective training and infe…”

DeepSeek-V3: Achieving Efficient LLM Scaling with 2,048 GPUs原评论 ↗
评论lexandstuff · 2025-05

“Except that, all Deepseek models so far have been trained on Nvidia hardware. For Deepseek v3, they literally mention that they used 2,048 NVIDIA H800 GPUs right in the abstract: https://arxiv.org/html/2505.09343v1”

Deepseek R1-0528原评论 ↗
评论lhl · 2025-08

“A few people have mentioned looking a the vLLM docs and blog (recommended!). I'd also recommend SGLang's docs and blog as well. If you're interested in a bit of a deeper dive, I can highly recommend reading some of what DeepSeek has published: https://arxiv.org/abs/2505.09343 (and actually quite a few of their Technical Reports and papers). I'd also say…”

Ask HN: How can ChatGPT serve 700M users when I can't run one GPT-4 locally?原评论 ↗
被提交为 story · 4 次