引用estreeper · 2023-11
“the extent to which the models are able to ignore irrelevant context, which is a problem [2].”
指出模型能否忽略无关上下文是个问题,引它为证
加入无关信息会让大模型的数学推理准确率大幅下降
论文研究大语言模型的“易分心性”,即无关上下文会如何影响解题准确率。作者构造了在题面中带有无关信息的小学数学推理数据集 GSM-IC,用它测试当时先进的提示技术,发现一旦加入无关信息,模型表现就大幅下降。他们还找到几种缓解办法,如自洽解码,以及在提示中明确要求模型忽略无关信息。
评论者引用它说明大模型会被提示中的无关上下文干扰,用来反驳“上下文越多越好”的说法。
“the extent to which the models are able to ignore irrelevant context, which is a problem [2].”
“Trivially falsifiable:”