按年被提到
20212022 · 1 次2026
大模型与语言模型›量化、解码与分词·arXiv:2109.05093 · cs.CL

PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models

Torsten Scholak, Nathan Schucher, Dzmitry Bahdanau · 2021
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2022-03-19 · 最近 2026-09-23
2不同的人
2活跃月份
2讨论串
405被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Large pre-trained language models for textual data have an unconstrained output space; at each decoding step, they can produce any of 10,000s of sub-word tokens. When fine-tuned to target constrained formal languages like SQL, these models often generate invalid code, rendering it unusable. We propose PICARD (code and trained models available at https://github.com/ElementAI/picard), a method for constraining auto-regressive decoders of language models through incremental parsing. PICARD helps to find valid output sequences by rejecting inadmissible tokens at each decoding step. On the challenging Spider and CoSQL text-to-SQL translation tasks, we show that PICARD transforms fine-tuned T5 models with passable performance into state-of-the-art solutions.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论bob1029 · 2022-03

“We tried using OpenAI/Davinci for SQL query authoring, but it quickly became obvious that we are still really far from something the business could find value in. The state of the art as described below is nowhere near where we would need it to be: https://yale-lily.github.io/spider https://arxiv.org/abs/2109.05093 https://github.com/ElementAI/picard …”

It’s Like GPT-3 but for Code – Fun, Fast, and Full of Flaws原评论 ↗
被提交为 story · 1 次