“Heres the Arxiv link with the PDF since the Science link isn't working. https://arxiv.org/pdf/2203.07814v1.pdf *See p. 21 for a list of benchmarks, including the results of a finetuned GPT-3 for coding.”
Competition-Level Code Generation with AlphaCode
Programming is a powerful and ubiquitous problem-solving tool. Developing systems that can assist programmers or even generate programs independently could make programming more productive and accessible, yet so far incorporating innovations in AI has proven challenging. Recent large-scale language models have demonstrated an impressive ability to generate code, and are now able to complete simple programming tasks. However, these models still perform poorly when evaluated on more complex, unseen problems that require problem-solving skills beyond simply translating instructions into code. For example, competitive programming problems which require an understanding of algorithms and complex natural language remain extremely challenging. To address this gap, we introduce AlphaCode, a system for code generation that can create novel solutions to these problems that require deeper reasoning. In simulated evaluations on recent programming competitions on the Codeforces platform, AlphaCode achieved on average a ranking of top 54.3% in competitions with more than 5,000 participants. We found that three key components were critical to achieve good and reliable performance: (1) an extensive and clean competitive programming dataset for training and evaluation, (2) large and efficient-to-sample transformer-based architectures, and (3) large-scale model sampling to explore the search space, followed by filtering based on program behavior to a small set of submissions.
候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“Nature paper: https://www.science.org/doi/10.1126/science.abq1158 For those wondering, yes, these are the same results reported in the February 2022 pre-print here: https://arxiv.org/abs/2203.07814 As in the preprint the authors report their system's average ranking as top 54.3% in past competitions but the meat and potatoes of their results are in th…”
“>> - Image descriptions eg. Be My Eyes: https://www.bemyeyes.com/ Not working yet. From the website: Starting today, you can register for the waitlist in the Be My Eyes app. >> - Customer service chat triage. More specifically? Which company is currently using LLMs for this purpose? >> - helping students learn- eg khan academy, tutor Lily Helping th…”
- Competition-Level Code Generation with AlphaCode ↗signa11 · 2022-12-09 · 1 分