按年被提到
20232023 · 1 次2026
大模型与语言模型›长上下文与测试时计算·arXiv:2311.06668 · cs.LG

In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steering

Sheng Liu, Haotian Ye, Lei Xing, James Zou · 2023
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2023-11-19 · 最近 2025-08-02
2不同的人
2活跃月份
2讨论串
49被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Large language models (LLMs) demonstrate emergent in-context learning capabilities, where they adapt to new tasks based on example demonstrations. However, in-context learning has seen limited effectiveness in many settings, is difficult to quantitatively control and takes up context window space. To overcome these limitations, we propose an alternative approach that recasts in-context learning as in-context vectors (ICV). Using ICV has two steps. We first use a forward pass on demonstration examples to create the in-context vector from the latent embedding of the LLM. This vector captures essential information about the intended task. On a new query, instead of adding demonstrations to the prompt, we shift the latent states of the LLM using the ICV. The ICV approach has several benefits: 1) it enables the LLM to more effectively follow the demonstration examples; 2) it's easy to control by adjusting the magnitude of the ICV; 3) it reduces the length of the prompt by removing the in-context demonstrations; 4) ICV is computationally much more efficient than fine-tuning. We demonstrate that ICV achieves better performance compared to standard in-context learning and fine-tuning on diverse tasks including safety, style transfer, role-playing and formatting. Moreover, we show that we can flexibly teach LLM to simultaneously follow different types of instructions by simple vector arithmetics on the corresponding ICVs.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论qrios · 2025-08

“Current "good enough" models like Mistral Small require GPUs like the RTX 6000 to achieve user-friendly response times. The model quality is good enough, especially for narrow-scope tasks like summarization and classification. If Moore's Law holds for a few more years, a mobile device will be able to run it on-device in around 8 years (Apple's A11: 410 GF…”

Tim Cook rallying Apple employees around AI efforts原评论 ↗
被提交为 story · 1 次