按年被提到
20222022 · 4 次2026
软件工程与 AI 编程›代码生成与程序合成·arXiv:2207.01780 · cs.LG

CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learning

Hung Le, Yue Wang, Akhilesh Deepak Gotmare, Silvio Savarese, Steven C. H. Hoi · 2022
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2022-07-21 · 最近 2022-08-07
2不同的人
2活跃月份
3讨论串
161被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Program synthesis or code generation aims to generate a program that satisfies a problem specification. Recent approaches using large-scale pretrained language models (LMs) have shown promising results, yet they have some critical limitations. In particular, they often follow a standard supervised fine-tuning procedure to train a code generation model only from the pairs of natural-language problem descriptions and ground-truth programs. Such paradigm largely ignores some important but potentially useful signals in the problem specification such as unit tests, which thus often results in poor performance when solving complex unseen coding tasks. To address the limitations, we propose "CodeRL", a new framework for program synthesis tasks through pretrained LMs and deep reinforcement learning (RL). Specifically, during training, we treat the code-generating LM as an actor network, and introduce a critic network that is trained to predict the functional correctness of generated programs and provide dense feedback signals to the actor. During inference, we introduce a new generation procedure with a critical sampling strategy that allows a model to automatically regenerate programs based on feedback from example unit tests and critic scores. For the model backbones, we extended the encoder-decoder architecture of CodeT5 with enhanced learning objectives, larger model sizes, and better pretraining data. Our method not only achieves new SOTA results on the challenging APPS benchmark, but also shows strong zero-shot transfer capability with new SOTA results on the simpler MBPP benchmark.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论moyix · 2022-07

“The TL;DR is that this appears to beat both OpenAI Codex (12 billion params) and DeepMind's AlphaCode (1 billion params), despite having only 770M parameters. There is also a paper: https://arxiv.org/abs/2207.01780 And the code is available: https://github.com/salesforce/CodeRL”

AI Coding with CodeRL: Toward Mastering Program Synthesis with Deep RL原评论 ↗
评论moyix · 2022-07

“Reposting my commentary: The TL;DR is that this appears to beat both OpenAI Codex (12 billion params) and DeepMind's AlphaCode (1 billion params), despite having only 770M parameters. There is also a paper: https://arxiv.org/abs/2207.01780 And the code is available: https://github.com/salesforce/CodeRL So this seems to buck the trend of needing enormou…”

AI Coding with CodeRL: Toward Mastering Program Synthesis with Deep RL原评论 ↗
评论YeGoblynQueenne · 2022-08

“>> Do you remember how people on HN used to take comfort that AI couldn't write code? Now they're taking comfort in the fact that AI can't write code well. If you're referring to neural program synthesis with large language models (LLMs), the performance of current systems is pitifully low, with best results in the range of 5-7% correct programs when eva…”

AlphaFold won’t revolutionise drug discovery原评论 ↗
评论YeGoblynQueenne · 2022-08

“Where is this claim from? The Codex paper claims 28.8% on OpenAI's novel dataset: On HumanEval, a new evaluation set we release to measure functional correctness for synthesizing programs from docstrings, our model solves 28.8% of the problems, while GPT-3 solves 0% and GPT-J solves 11.4%. https://arxiv.org/abs/2107.03374 The numbers I quote (5-7%) are…”

AlphaFold won’t revolutionise drug discovery原评论 ↗