按年被提到
20232023 · 2 次2026
大模型与语言模型›提示工程与输出稳定性·arXiv:2309.09128 · cs.HC

ChainForge: A Visual Toolkit for Prompt Engineering and LLM Hypothesis Testing

Ian Arawjo, Chelse Swoopes, Priyan Vaithilingam, Martin Wattenberg, Elena Glassman · 2023
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2023-09-19 · 最近 2024-12-21
2不同的人
2活跃月份
3讨论串
161被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Evaluating outputs of large language models (LLMs) is challenging, requiring making -- and making sense of -- many responses. Yet tools that go beyond basic prompting tend to require knowledge of programming APIs, focus on narrow domains, or are closed-source. We present ChainForge, an open-source visual toolkit for prompt engineering and on-demand hypothesis testing of text generation LLMs. ChainForge provides a graphical interface for comparison of responses across models and prompt variations. Our system was designed to support three tasks: model selection, prompt template design, and hypothesis testing (e.g., auditing). We released ChainForge early in its development and iterated on its design with academics and online users. Through in-lab and interview studies, we find that a range of people could use ChainForge to investigate hypotheses that matter to them, including in real-world settings. We identify three modes of prompt engineering and LLM hypothesis testing: opportunistic exploration, limited evaluation, and iterative refinement.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论pgbovine · 2024-12

“so exciting to see these ideas out in the world! i'm now imagining a Scratch-like playground for kids to explore end-user programming / AI in an accessible way, like some of the example apps you've shown there's been a rising tide of academic HCI work in a similar space, wonder if there will be cross-pollination of ideas along these lines (many more pape…”

Tldraw Computer原评论 ↗
被提交为 story · 2 次