按年被提到
20222022 · 1 次2026
大模型与语言模型›量化、解码与分词·arXiv:2205.01068 · cs.CL

OPT: Open Pre-trained Transformer Language Models

Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, Todor Mihaylov, Myle Ott 等 19 人 · 2022
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2022-05-03 · 最近 2026-09-04
2不同的人
2活跃月份
2讨论串
2,026被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Large language models, which are often trained for hundreds of thousands of compute days, have shown remarkable capabilities for zero- and few-shot learning. Given their computational cost, these models are difficult to replicate without significant capital. For the few that are available through APIs, no access is granted to the full model weights, making them difficult to study. We present Open Pre-trained Transformers (OPT), a suite of decoder-only pre-trained transformers ranging from 125M to 175B parameters, which we aim to fully and responsibly share with interested researchers. We show that OPT-175B is comparable to GPT-3, while requiring only 1/7th the carbon footprint to develop. We are also releasing our logbook detailing the infrastructure challenges we faced, along with code for experimenting with all of the released models.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论kbwal7 · 2026-09

“It's pretty incredible how robust neural networks are to even things like architectural changes mid-run. The team at Meta when training their own copy of GPT-3 (called OPT-3) even changed optimizers mid-run (from AdamW -> SGD -> AdamW, https://arxiv.org/pdf/2205.01068 section 2.5)!”

Marin 32B Spike Fixing原评论 ↗
被提交为 story · 1 次