“OpenAI's "GPT-4 Technical Report" doesn't say much more than the fact that it's "Transformer-based".”
GPT-4 Technical Report
OpenAI 对多模态 GPT-4 的能力评测与安全报告,细节有限
报告介绍 GPT-4:一个可接受图像和文本输入、输出文本的大规模多模态模型,在多项专业和学术基准上达到人类水平,例如模拟律师资格考试成绩约在前 10%。它是基于 Transformer、以预测下一个 token 预训练的模型,后训练对齐提升了事实性与行为合规性。项目的核心之一是构建能跨尺度可预测地扩展的基础设施和优化方法,因此可用不到 GPT-4 千分之一算力训练的模型预测其部分性能。
评论者主要把它当作引用来源:查 GPT-4 的考试成绩和基准数字、RLHF 的影响、ARC 红队测试案例(如骗 TaskRabbit 工人解验证码),以及 OpenAI 未披露架构和训练数据这一点。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“They argue in the appendix that it does not affect 'capability' in answering test questions. But, there is a confounding factor.”
“This does not seem a serious decontamination effort. It should not give confidence that the data is actually properly deduplicated/decontaminated.”
“but that doesn't really clear up how the images are being treated here.”
“FWIW, the “HellaSwag” benchmark from the GPT-4 paper consists of many such “riddles” and it achieves ~95% accuracy (essentially par with humans).”
“Actually, they have release some details about it, in this 99-page technical report GPT-4 Technical Report”
看全部 20 条 →
“In Appendix B[0], they show that RLHF does not improve capabilities on human tasks.”
“Common one right now is to give LLMs the same exams we put humans through, leading to e.g. the graph on page 6: GPT-4 Technical Report”
“GPT-4 paper, page 10: GPT-4 Technical Report”
“Overreliance occurs when users excessively trust and depend on the model, potentially leading to unnoticed mistakes and inadequate oversight.”
“It’s in their latest paper here: GPT-4 Technical Report”
“these are maybe some good starting points”
“The technical report has before and after comparisons.”
“(from: GPT-4 paper: GPT-4 Technical Report)”
“Overreliance is a failure mode that likely increases with model capability and reach.”
“The following is an illustrative example of a task that ARC conducted using the model:”
“Also in the little that OpenAI published about GPT4, i believe one of the examples went from unaligned racism against black people to aligned mild racism against white people.”
“GPT4 technical paper doesn't seem to disclose it (or I didn't dive deep enough): GPT-4 Technical Report”
“the Technical Report says”
“The report cites both GPT-3.5 and GPT-4 scores on page 7 [1].”