按年被提到
20232023 · 26 次2026
大模型与语言模型›大模型预训练与缩放·arXiv:2302.04761 · cs.CL

Toolformer: Language Models Can Teach Themselves to Use Tools

Toolformer:语言模型可以自学使用工具
Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, Thomas Scialom · 2023

语言模型以自监督方式学会何时调用外部 API

开始阅读 ↗PDFalphaXiv 讨论
被推荐 · 20 人首次 2023-02-10 · 最近 2023-11-23
20不同的人
5活跃月份
22讨论串
1,682被引(OpenAlex)

提出 Toolformer:一个以自监督方式训练、学会决定调用哪些 API、何时调用、传什么参数,以及如何把返回结果用于后续 token 预测的模型,每个 API 只需少量示范。集成的工具包括计算器、问答系统、两种搜索引擎、翻译系统和日历。Toolformer 在多种下游任务上零样本表现大幅提升,往往可与大得多的模型竞争,且不牺牲核心语言建模能力。

HN 为什么提它

评论者围绕它讨论小模型也能把人的意图转成工具调用,并欣赏它在训练中试调 API、比较下一词概率来生成微调数据的做法;多出现在 ChatGPT 类产品讨论里。

方法难度 3 / 5约 45 分钟有时效性
读之前最好知道
1大语言模型与微调
2自监督学习
3上下文学习(in-context learning)
4外部 API 与检索增强基础

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
推荐localhost · 2023-02

“Relevant to this discussion is the Toolformer paper that was just released today from Meta AI. To me the exciting result from the paper is that it appears that much smaller models can be used to translate human intent into actions:”

推荐 Toolformer,令人兴奋的是小得多的模型也能把人类意图转成行动
A new way to build with Large Language Models原评论 ↗
推荐valine · 2023-02

“Toolformer does something really neat where they make the API call during training and compare next word probability of the API result with the generated result.”

称赞它训练时调用 API,并比较 API 结果与生成结果的下一词概率,很巧妙
Bing AI Can't Be Trusted原评论 ↗
推荐valine · 2023-02

“Checkout toolformer, they’ve got this mostly working with a much smaller model than gpt3.5.”

推荐看 Toolformer,用比 GPT-3.5 小得多的模型基本实现了这一点
Text Is All You Need原评论 ↗
推荐williamcotton · 2023-02

“The authors show a valuable method of using prompts to generate training data to be used for fine-tuning.”

推荐它用提示词生成微调训练数据的方法,示例提示在附录 A.2
Prompt Engineering Guide: Guides, papers, and resources for prompt engineering原评论 ↗
推荐williamcotton · 2023-02

“Here’s some recent ML work from Meta that I really like:”

列为自己很喜欢的 Meta 近期 ML 工作,讲语言模型自学使用工具
Zuckerberg: We're creating a top-level group at Meta focused on generative AI原评论 ↗
推荐ninjin · 2023-03

“A recent favourite would be the Toolformer paper [1], that I intend to read in detail later today.”

称这是自己近期的最爱,打算细读,讲 LLM 使用外部工具
ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks原评论 ↗
看全部 20 条 →
推荐Version467 · 2023-04

“I can recommend the Sparks of AGI paper[2] and the toolformer paper[3].”

推荐 Sparks of AGI 和这篇 Toolformer 论文
Saying Goodbye to GitHub – Ersei 'N Stuff原评论 ↗
引用btown · 2023-02

“See, for instance, [LINK] / Toolformer where the language model is used to create training data that allows it to emit tokens that function as lookup oracles by interacting with external APIs.”

举例说明语言模型可自造训练数据,学会通过外部 API 做查询
Bing AI Can't Be Trusted原评论 ↗
引用danielbln · 2023-02

“the use of external tools[1], also see toolformerzero[2] for a simple example.”

指出用外部工具是提升 LLM 事实性的办法,并以它为例
Our Society Is Not Prepared for This Much AI Awesome原评论 ↗
引用williamcotton · 2023-02

“Third-order, Toolformer: Toolformer”

列为第三层次的例子,LLM 学会调用外部工具 API
Prompt Engineering Guide: Guides, papers, and resources for prompt engineering原评论 ↗
引用BoorishBears · 2023-02

“There's research being done on this: Toolformer”

指出已有研究让语言模型学会用工具来处理事实类任务
ChatGPT is falsely recommending us for a service we don't provide原评论 ↗
引用imranq · 2023-02

“which uses APIs and functions to improve GPT-J beyond GPT-3 for various tasks”

引它说明借助 API 和函数,GPT-J 在多种任务上超过 GPT-3
Meta rolls out AI language model LLaMA原评论 ↗
引用jackblemming · 2023-03

“Toolformer: Toolformer”

列入资源清单,讲 LLM 学会调用外部工具
Ask HN: Are people using ChatGPT to run things?原评论 ↗
引用jkbyc · 2023-03

“That's called Toolformer (Facebook AI Research): Toolformer”

指出这就是 Facebook AI Research 的 Toolformer,让语言模型自学调用工具
ChatGPT Explained: A Normie's Guide to How It Works原评论 ↗
引用sprobertson · 2023-03

“He likely meant techniques such as ToolFormer[1] where the language model will output a "request" (in some text syntax) that another system can parse, run and report back.”

解释对方可能指 Toolformer 这类技术:模型输出请求,由其他系统解析执行并回报
引用goldwelder42 · 2023-03

“GPT-4 is already using some variant of toolformer”

推测 GPT-4 已在用某种 Toolformer 式的变体
ViperGPT: A Python Framework That Combines Vision and Language Models Using Code原评论 ↗
引用wcoenen · 2023-03

“a paper about how to hook up LLMs to any external tool Toolformer”

推荐一篇讲如何把 LLM 接到任何外部工具的论文
ViperGPT: A Python Framework That Combines Vision and Language Models Using Code原评论 ↗
引用johntb86 · 2023-04

“This document seems to have been written before the toolformer paper[0], which fine tunes the model to use tools (e.g search) to retrieve information.”

指出该文档似乎写在 Toolformer 之前,后者微调模型去调用搜索等工具获取信息
Knowledge retrieval architectures for LLMs (2023)原评论 ↗
引用og_kalu · 2023-05

“It should be in line with the likes of this - Toolformer or [LINK]”

指出应与此类工作同路,即模型自己学会何时调用外部工具
AI knows things no one told it原评论 ↗
引用seanhunter · 2023-05

“That’s sort of what happens in the “toolformer” paper[1] which sets out the basic architecture by which things like chatGPT plugins work.”

称它给出了 ChatGPT 插件这类东西的基本架构
Meta AI Unleashes Megabyte, a Revolutionary Scalable Model Architecture原评论 ↗