“Relevant to this discussion is the Toolformer paper that was just released today from Meta AI. To me the exciting result from the paper is that it appears that much smaller models can be used to translate human intent into actions:”
Toolformer: Language Models Can Teach Themselves to Use Tools
语言模型以自监督方式学会何时调用外部 API
提出 Toolformer:一个以自监督方式训练、学会决定调用哪些 API、何时调用、传什么参数,以及如何把返回结果用于后续 token 预测的模型,每个 API 只需少量示范。集成的工具包括计算器、问答系统、两种搜索引擎、翻译系统和日历。Toolformer 在多种下游任务上零样本表现大幅提升,往往可与大得多的模型竞争,且不牺牲核心语言建模能力。
评论者围绕它讨论小模型也能把人的意图转成工具调用,并欣赏它在训练中试调 API、比较下一词概率来生成微调数据的做法;多出现在 ChatGPT 类产品讨论里。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“Toolformer does something really neat where they make the API call during training and compare next word probability of the API result with the generated result.”
“Checkout toolformer, they’ve got this mostly working with a much smaller model than gpt3.5.”
“The authors show a valuable method of using prompts to generate training data to be used for fine-tuning.”
“Here’s some recent ML work from Meta that I really like:”
“A recent favourite would be the Toolformer paper [1], that I intend to read in detail later today.”
看全部 20 条 →
“I can recommend the Sparks of AGI paper[2] and the toolformer paper[3].”
“See, for instance, [LINK] / Toolformer where the language model is used to create training data that allows it to emit tokens that function as lookup oracles by interacting with external APIs.”
“the use of external tools[1], also see toolformerzero[2] for a simple example.”
“Third-order, Toolformer: Toolformer”
“There's research being done on this: Toolformer”
“which uses APIs and functions to improve GPT-J beyond GPT-3 for various tasks”
“Toolformer: Toolformer”
“That's called Toolformer (Facebook AI Research): Toolformer”
“He likely meant techniques such as ToolFormer[1] where the language model will output a "request" (in some text syntax) that another system can parse, run and report back.”
“GPT-4 is already using some variant of toolformer”
“a paper about how to hook up LLMs to any external tool Toolformer”
“This document seems to have been written before the toolformer paper[0], which fine tunes the model to use tools (e.g search) to retrieve information.”
“It should be in line with the likes of this - Toolformer or [LINK]”
“That’s sort of what happens in the “toolformer” paper[1] which sets out the basic architecture by which things like chatGPT plugins work.”