“I'm leaving this for further reading. It's fascinating.”
Evolution through Large Models
用代码大模型当遗传编程的变异算子,生成海量程序
论文的出发点是:经代码生成训练的大语言模型能大幅提升遗传编程中程序变异算子的效果,因为它们的训练数据包含一系列连续修改,能近似人类可能做出的改动。主实验中,这种“大模型进化”(ELM)结合 MAP-Elites,在 Sodarace 领域生成了数十万个能输出可行走机器人的功能性 Python 程序,而原 LLM 在预训练中从未见过这类程序。这些样本再用来引导训练一个新的条件语言模型,使其能为特定地形输出合适的行走器,作者还深入探讨了对开放性、深度学习和强化学习的影响。
HN 上的人在讨论用 LLM 做变异算子的进化式搜索、基于测试验证的反馈学习以及 AlphaEvolve 类工作时,反复把它当作代表性的早期例子。
学习路径
读这篇之前,按顺序读书架上的 2 篇 · 参考文献来自 OpenAlex,顺序和要点由 Claude Code 排出只有两篇候选与目标直接相关:Transformer 提供语言模型基础,AlphaCode 展示代码生成。读完后再看大模型如何充当遗传编程的变异算子。
- 1
- 2
它展示了大模型生成程序的能力和采样筛选思路,是 ELM 使用代码模型的背景。
读大规模采样再过滤的流程,以及模型生成程序的质量与多样性。
- 3Evolution through Large Models
把代码大模型当成遗传编程的变异算子,与 MAP-Elites 结合生成大量程序。
读差异(diff)形式的变异算子、MAP-Elites 质量多样性搜索的思路,以及机器人行走实验。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“The paper "Evolution through Large Models" shows the way.”
“Here is a related paper:”
“Here is an example of using code language models for this kind of dataset-creation by play.”
“For example: "Evolution through Large Models" Evolution through Large Models”
“It is true that simply using a dataset or a bunch of environments is not enough.”
看全部 19 条 →
“In their paper "Evolution through Large Models", CarperAI shows how you can use diffs to evolve code, while running the code against a test or an environment for validation.”
“LLMs can also learn by reinforcement learning and evolutionary techniques.”
“They are using a diff-LM trained on GitHub commits to generate mutations to a population of problem solutions expressed as code.”
“when they get tools they can integrate external signals and actually become creative and even discover knowledge - like [1]”
“Here is a paper doing evolutionary approaches on top of LLM generating code.”
“For example: Evolution through Large Models - Evolution through Large Models”
“They can also generate the model code, explain it, make meaningful changes[2], monitor the training run and evaluate the "child" models[3].”
“Evolution through Large Models - Evolution through Large Models”
“You can do pretty amazing things with evolutionary methods where the LLMs are the mutation operator.”
“Here is an example of LLMs discovering useful ideas by feedback, even when they are completely outside their training distribution:”
“In other words, the model learns to predict plausible changes to code from examples of changes made to code by human programmers.”
“written in collaboration between OpenAI and Anthropic researchers”
“And some academic papers kind of in this space: Evolution through Large Models, [OTHER_PAPER], [OTHER_PAPER]”
- Evolution Through Large Models ↗lnyan · 2022-06-20 · 32 分