按年被提到
20232023 · 4 次2026
大模型与语言模型›大模型推理与思维链·arXiv:2308.00304 · cs.CL

Skills-in-Context Prompting: Unlocking Compositionality in Large Language Models

Jiaao Chen, Xiaoman Pan, Dian Yu, Kaiqiang Song, Xiaoyang Wang, Dong Yu, Jianshu Chen · 2023
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2023-08-02 · 最近 2025-05-31
2不同的人
4活跃月份
5讨论串
12被引(OpenAlex)
arXiv 摘要(候选层不写导读)

We investigate how to elicit compositional generalization capabilities in large language models (LLMs). Compositional generalization empowers LLMs to solve complex problems by combining foundational skills, a critical reasoning ability akin to human intelligence. However, even the most advanced LLMs currently struggle with this form of reasoning. We examine this problem within the framework of in-context learning and find that demonstrating both foundational skills and compositional examples grounded in these skills within the same prompt context is crucial. We refer to this prompt structure as skills-in-context (SKiC). With as few as two exemplars, this in-context learning structure enables LLMs to tackle more challenging problems requiring innovative skill combinations, achieving near-perfect systematic generalization across a broad range of tasks. Intriguingly, SKiC also unlocks the latent potential of LLMs, allowing them to more actively utilize pre-existing internal skills acquired during earlier pretraining stages to solve complex reasoning problems. The SKiC structure is robust across different skill constructions and exemplar choices and demonstrates strong transferability to new tasks. Finally, inspired by our in-context learning study, we show that fine-tuning LLMs with SKiC-style data can elicit zero-shot weak-to-strong generalization, enabling the models to solve much harder problems directly with standard prompting.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论og_kalu · 2023-08

“LLMs are a different beast in the ML world. Finetuned Palm for Medicine and Finetuned Minerva for Math all perform a good deal worse than GPT-4. A fine-tuned smaller model is by no means guaranteed to beat a larger more general one (though of course you may get acceptable performance). And then the necessity of fine-tuning itself is called into questio…”

You probably don’t need to fine-tune LLMs原评论 ↗
评论og_kalu · 2023-10

“> Numbers (LLMs still can't do math and the ones that "can" are very brute forced and don't generalize) These seem to generalize. https://arxiv.org/abs/2211.09066 https://arxiv.org/abs/2308.00304 > Commutative properties (The (idk why anyone was surprised) "reversal curse": https://arxiv.org/abs/2309.12288) This was a retrieval from training issue no…”

LLMs can't self-correct in reasoning tasks, DeepMind study finds原评论 ↗
评论og_kalu · 2023-12

“>Every human is not even capable of reading, or speaking, or thinking at all. But those humans who can't are not the benchmark of general human intelligence. It's simple. If you're going to disqualify a potential general intelligence because of a particular task solving ability, it better be something every general intelligence can do. Now let's forget …”

Weak-to-Strong Generalization原评论 ↗
评论neom · 2025-05

“the extraordinary claim would be that LLMs can only do things they've seen before exactly, given the compositional and emergent capabilities we observe. The evidence suggests they can generalize beyond their training in meaningful ways, even if imperfectly...if a human came out living but with a brain that had zero electrical activity, that would be extra…”

Surprisingly fast AI-generated kernels we didn't mean to publish yet原评论 ↗
被提交为 story · 1 次