按年被提到
20242024 · 2 次2026
大模型与语言模型›大模型预训练与缩放·arXiv:2401.16818 · cs.CL

H2O-Danube-1.8B Technical Report

Philipp Singer, Pascal Pfeiffer, Yauhen Babakhin, Maximilian Jeblick, Nischay Dhankhar, Gabor Fodor, Sri Satish Ambati · 2024
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2024-04-05 · 最近 2026-04-30
2不同的人
2活跃月份
3讨论串
6被引(OpenAlex)
arXiv 摘要(候选层不写导读)

We present H2O-Danube, a series of small 1.8B language models consisting of H2O-Danube-1.8B, trained on 1T tokens, and the incremental improved H2O-Danube2-1.8B trained on an additional 2T tokens. Our models exhibit highly competitive metrics across a multitude of benchmarks and, as of the time of this writing, H2O-Danube2-1.8B achieves the top ranking on Open LLM Leaderboard for all models below the 2B parameter range. The models follow core principles of LLama 2 and Mistral, and we leverage and refine various techniques for pre-training large language models. We additionally release chat models trained with supervised fine-tuning followed by direct preference optimization. We make all models openly available under Apache 2.0 license further democratizing LLMs to a wider audience economically.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论nickpsecurity · 2026-04

“On changing the training mix, H20 did that with Danube in 2024: https://arxiv.org/pdf/2401.16818 With those results, I would've already done that in any models I got to train. There's also the principle that the LLM's are often better at what they saw last in their training set. That also justifies putting more logic, code, and math in at the end for an…”

Granite 4.1: IBM's 8B Model Matching 32B MoE原评论 ↗
被提交为 story · 2 次