按年被提到
20222023 · 3 次2026
大模型与语言模型›大模型预训练与缩放·arXiv:2211.05100 · cs.CL

BLOOM: A 176B-Parameter Open-Access Multilingual Language Model

BigScience Workshop, :, Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilić, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé 等 394 人 · 2022
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 3 人首次 2023-04-11 · 最近 2023-05-23
3不同的人
2活跃月份
3讨论串
1,593被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to widespread adoption, most LLMs are developed by resource-rich organizations and are frequently kept from the public. As a step towards democratizing this powerful technology, we present BLOOM, a 176B-parameter open-access language model designed and built thanks to a collaboration of hundreds of researchers. BLOOM is a decoder-only Transformer language model that was trained on the ROOTS corpus, a dataset comprising hundreds of sources in 46 natural and 13 programming languages (59 in total). We find that BLOOM achieves competitive performance on a wide variety of benchmarks, with stronger results after undergoing multitask prompted finetuning. To facilitate future research and applications using LLMs, we publicly release our models and code under the Responsible AI License.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论espadrine · 2023-04

“We can infer from publicly available information. BLOOM[0] was trained for four months on 384 A100 80GB GPUs, excluding architecture search. They specifically indicate (in the Huggingface page): > Estimated cost of training: Equivalent of $2-5M in cloud computing (including preliminary experiments) You can see from the training loss[1] that it was still…”

Anthropic’s $5B, 4-year plan to take on OpenAI原评论 ↗
评论visarga · 2023-05

“No, the Bloom paper is wild: 2.5 pages of author names written with no spacing, I counted 472 authors after deduplication. PaLM 2 has only 181 authors.;) https://arxiv.org/pdf/2211.05100.pdf”

RWKV: Reinventing RNNs for the Transformer Era原评论 ↗
被提交为 story · 1 次