按年被提到
20232023 · 2 次2026
大模型与语言模型›长上下文与测试时计算·arXiv:2307.04721 · cs.AI

Large Language Models as General Pattern Machines

Suvir Mirchandani, Fei Xia, Pete Florence, Brian Ichter, Danny Driess, Montserrat Gonzalez Arenas, Kanishka Rao, Dorsa Sadigh, Andy Zeng · 2023
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2023-07-11 · 最近 2023-08-20
2不同的人
2活跃月份
2讨论串
102被引(OpenAlex)
arXiv 摘要(候选层不写导读)

We observe that pre-trained large language models (LLMs) are capable of autoregressively completing complex token sequences -- from arbitrary ones procedurally generated by probabilistic context-free grammars (PCFG), to more rich spatial patterns found in the Abstraction and Reasoning Corpus (ARC), a general AI benchmark, prompted in the style of ASCII art. Surprisingly, pattern completion proficiency can be partially retained even when the sequences are expressed using tokens randomly sampled from the vocabulary. These results suggest that without any additional training, LLMs can serve as general sequence modelers, driven by in-context learning. In this work, we investigate how these zero-shot capabilities may be applied to problems in robotics -- from extrapolating sequences of numbers that represent states over time to complete simple motions, to least-to-most prompting of reward-conditioned trajectories that can discover and represent closed-loop policies (e.g., a stabilizing controller for CartPole). While difficult to deploy today for real systems due to latency, context size limitations, and compute costs, the approach of using LLMs to drive low-level control may provide an exciting glimpse into how the patterns among words could be transferred to actions.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

没有评论直接链到它;下面是它被提交的讨论串里回复最多的评论(不计入统计)
讨论串评论asklotsogque · 2023-08

“Amazing how much the mainstream media are still obsessed with LLMs and factual accuracy. I prefer to see them as the worlds greatest innovation as lateral thinking machines ie pattern A applied over pattern B to give a credible pattern C.”

Large Language Models As General Pattern Machines原评论 ↗
讨论串评论golol · 2023-08

“This is the real breakthrough of LLMs imo. Karpathy had a series of tweest about this a year ago. It makes sense: Prettained LLMs learn to continue sequences of tokens. These are language tokens but the language is so complex and rich that a kind of general pattern recognition and continuation ability is learned. I think finetuning and alignment destroys …”

Large Language Models As General Pattern Machines原评论 ↗
讨论串评论GaggiX · 2023-08

“Looking at the ARC problems that the model didn't solve correctly I honestly have no idea in some of done on how the model of wrong or what should have been the solution given the train example. How strong is a human being on this challenge?”

Large Language Models As General Pattern Machines原评论 ↗
被提交为 story · 2 次