引用evrydayhustling · 2019-05
“DNNs encode incredibly specific histories of training data in their weights”
拿它说明 DNN 的权重编码了训练数据的具体历史
量化生成式序列模型无意记住罕见训练序列的风险并提取秘密
论文提出一套测试方法,定量评估生成式序列模型无意中记住罕见或唯一训练序列的风险,让从业者能选择尽量减少记忆的训练方式。实验表明,非预期记忆是一个持续存在、难以避免的问题,对未考虑记忆问题训练的模型,作者给出了能高效提取信用卡号等唯一秘密序列的新方法,并说明该测试可作为实用的第一道防线,应用于 Google Smart Compose 以量化限制数据暴露。
HN 上的人引用它说明神经网络会记住并可能泄露罕见的训练数据,例如 Gmail Smart Compose 用私人邮件训练带来的隐私风险。
“DNNs encode incredibly specific histories of training data in their weights”
“It actually can be more pernicious than that: The Secret Sharer”
“For example Gmail's Smart Compose is trained on users' private messages”
“more powerful models can memorize and expose parts of their training data. See for example: The Secret Sharer”
“Models can unpredictably memorize sensitive input data, so there can be a real copyright issue here, I think.”