按年被提到
20232023 · 2 次2026
软件工程与 AI 编程›代码生成与程序合成·arXiv:2305.06161 · cs.CL

StarCoder: may the source be with you!

Raymond Li, Loubna Ben Allal, Yangtian Zi, Niklas Muennighoff, Denis Kocetkov, Chenghao Mou, Marc Marone, Christopher Akiki, Jia Li, Jenny Chim, Qian Liu, Evgenii Zheltonozhskii 等 67 人 · 2023
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2023-05-15 · 最近 2023-07-26
2不同的人
2活跃月份
2讨论串
727被引(OpenAlex)
arXiv 摘要(候选层不写导读)

The BigCode community, an open-scientific collaboration working on the responsible development of Large Language Models for Code (Code LLMs), introduces StarCoder and StarCoderBase: 15.5B parameter models with 8K context length, infilling capabilities and fast large-batch inference enabled by multi-query attention. StarCoderBase is trained on 1 trillion tokens sourced from The Stack, a large collection of permissively licensed GitHub repositories with inspection tools and an opt-out process. We fine-tuned StarCoderBase on 35B Python tokens, resulting in the creation of StarCoder. We perform the most comprehensive evaluation of Code LLMs to date and show that StarCoderBase outperforms every open Code LLM that supports multiple programming languages and matches or outperforms the OpenAI code-cushman-001 model. Furthermore, StarCoder outperforms every model that is fine-tuned on Python, can be prompted to achieve 40\% pass@1 on HumanEval, and still retains its performance on other programming languages. We take several important steps towards a safe open-access model release, including an improved PII redaction pipeline and a novel attribution tracing tool, and make the StarCoder models publicly available under a more commercially viable version of the Open Responsible AI Model license.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论sestinj · 2023-07

“We've experimented with it quite a bit, and at one point used it as our default model for inline edits. As they discuss in section 5.1 of the paper (https://arxiv.org/pdf/2305.06161.pdf), they've trained on data formatted as "<commit_before>code<commit_msg>text<commit_after>code<eos>", which was convenient for our use case. Unfortunately, it tended to rep…”

Show HN: Continue (YC S23) – Open-source coding autopilot原评论 ↗
被提交为 story · 1 次