按年被提到
20232023 · 2 次2026
大模型与语言模型›序列模型与注意力机制·arXiv:2307.09702 · cs.CL

Efficient Guided Generation for Large Language Models

Brandon T. Willard, Rémi Louf · 2023
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2023-08-14 · 最近 2024-05-25
2不同的人
2活跃月份
2讨论串
175被引(OpenAlex)
arXiv 摘要(候选层不写导读)

In this article we show how the problem of neural text generation can be constructively reformulated in terms of transitions between the states of a finite-state machine. This framework leads to an efficient approach to guiding text generation with regular expressions and context-free grammars by allowing the construction of an index over a language model's vocabulary. The approach is model agnostic, allows one to enforce domain-specific knowledge and constraints, and enables the construction of reliable interfaces by guaranteeing the structure of the generated text. It adds little overhead to the token sequence generation process and significantly outperforms existing solutions. An implementation is provided in the open source Python library Outlines

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论remilouf · 2023-08

“Outlines is a Python library that focuses on text generation with large language models. Brandon and I are not LLM experts and started the project a few months ago because we wanted to understand better how the generation process works. Our original background is probabilistic, relational and symbolic programming. Recently we came up with a fast way to g…”

Show HN: LLMs can generate valid JSON 100% of the time原评论 ↗
评论remilouf · 2023-08

“Figure 2 in our paper (https://arxiv.org/abs/2307.09702) shows the difference between guidance and outlines to generate a sequence that is valid to a regex. Jsonformer uses the same technique as guidance. Extrapolate this to several fields. Note that we still need to manage the KV cache in outlines. It’s a small interface change that will be made this we…”

Show HN: LLMs can generate valid JSON 100% of the time原评论 ↗
评论tikhonj · 2024-05

“Looks like it actually changes how you do token generation to conform to a given context-free grammar. It's a way to structure how you sample from the model rather than a tweak to the prompt, so it's more efficient and guarantees that the output matches the formal grammar. There's a reference to the paper that describes the method at the bottom of the RE…”

Mistral Fine-Tune原评论 ↗