引用WhitneyLand · 2023-12
“I think it’s valid work, but the original tweet seems to call a prompt based paper into question.”
认为这是有效的工作,但原推文似乎质疑了这篇基于提示词的论文
从开源到 ChatGPT 的语言模型中大规模提取训练数据
论文研究“可提取记忆”:对手无需了解训练集、仅靠查询模型就能高效提取的训练数据。作者在 Pythia、GPT-Neo 等开源模型,LLaMA、Falcon 等半开放模型以及 ChatGPT 等闭源模型上提取出数 GB 训练数据;针对已对齐的 ChatGPT,他们设计了新的发散攻击,使其脱离聊天式输出并以约 150 倍的速率吐出训练数据。结果说明实际攻击能恢复的数据远超此前认识,且当前的对齐技术并不能消除记忆。
评论者引用它说明文本模型同样会泄漏训练数据,尤其是让 ChatGPT 反复重复同一个词就会逐字吐出训练数据。
“I think it’s valid work, but the original tweet seems to call a prompt based paper into question.”
“Text, too: Scalable Extraction of Training Data from (Production) Language Models”
“get ChatGPT to regurgitate verbatim training data by asking it to repeat the same word over and over again[1]”
“I just have these ones off the top of my head”