“One of the authors, Charles, here. I wrote this up because 1) I've been following work on sampling/inference-time compute scaling and the "Large Language Monkeys" paper (https://arxiv.org/abs/2407.21787) results looked promising and 2) our reproduction was effortless and immediate, which has not been my experience with most research. Happy to answer ques…”
Large Language Monkeys: Scaling Inference Compute with Repeated Sampling
Scaling the amount of compute used to train language models has dramatically improved their capabilities. However, when it comes to inference, we often limit models to making only one attempt at a problem. Here, we explore inference compute as another axis for scaling, using the simple technique of repeatedly sampling candidate solutions from a model. Across multiple tasks and models, we observe that coverage -- the fraction of problems that are solved by any generated sample -- scales with the number of samples over four orders of magnitude. Interestingly, the relationship between coverage and the number of samples is often log-linear and can be modelled with an exponentiated power law, suggesting the existence of inference-time scaling laws. In domains like coding and formal proofs, where answers can be automatically verified, these increases in coverage directly translate into improved performance. When we apply repeated sampling to SWE-bench Lite, the fraction of issues solved with DeepSeek-Coder-V2-Instruct increases from 15.9% with one sample to 56% with 250 samples, outperforming the single-sample state-of-the-art of 43%. In domains without automatic verifiers, we find that common methods for picking from a sample collection (majority voting and reward models) plateau beyond several hundred samples and fail to fully scale with the sample budget.
候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“This power law behavior of test-time improvement seems to be pretty ubiquitous now. In more agents is all you need [1], they start to see this as a function of ensemble size. It also shows up in: Large Language Monkeys: Scaling Inference Compute with Repeated Sampling [2] I sorta wish everyone would plot their y-axis with logit y-axis, rather than 0->100…”
“Some other follow up reflections 1. I wish that Y-axes would switch to be logit instead of linear, to help see power-law scaling on these 0->1 measures. In this case, 20% -> 80% it doesn't really matter, but for other papers (eg. [2] below) it would help see this powerlaw behavior much better. 2. The power law behavior of inference compute seems to be s…”
“Just added o1 to https://double.bot if anyone would like to try it for coding. --- Some thoughts: * The performance is really good. I have a private set of questions I note down whenever gpt-4o/sonnet fails. o1 solved everything so far. * It really is quite slow * It's interesting that the chain of thought is hidden. This is I think the first time wh…”
“Amazing! OpenAI figured out how to scale inference. https://arxiv.org/abs/2407.21787 show how using more compute during inference can outperform much larger models in tasks like math problems I wonder how do they decide when to stop these Chain of Thought for each query? As anyone that played with agents can attest, LLMs can talk with themselves forever.”
- Beating GPT-4o and Claude 3.5 on SWE-bench Lite through repeated sampling ↗aglazer · 2024-08-01 · 5 分
- Large Language Monkeys: Scaling Inference Compute with Repeated Sampling ↗geuds · 2024-08-04 · 2 分