引用fernly · 2026-03
“Basically, they are so good at extracting clues from the text of the questions, and extrapolating from them, that they proceed to answer _as if_ they had an image to view.”
概述其发现:模型太会从题文里找线索,于是像有图一样作答
多模态模型没给图也能“看图作答”,暴露评测漏洞
论文报告三项发现:前沿多模态模型在根本没有提供图像时,仍会生成详细的图像描述和推理(作者称为 mirage reasoning);没有图像输入也能在通用和医学多模态基准上取得很高分,最极端时在一个胸片问答基准上排名第一;而被明确要求“猜答案”时成绩明显下降。作者据此呼吁使用排除文字线索的私有基准,并提出 B-Clean 用于基于视觉的公平评测。
评论者用它说明 LLM 会依据题面文字线索“假装看到了图”而作答,因此一些 AI 读片、医学影像评测的亮眼成绩可能并不可靠。
“Basically, they are so good at extracting clues from the text of the questions, and extrapolating from them, that they proceed to answer _as if_ they had an image to view.”
“See for example this recent paper where AI managed to beat radiologists on interpreting x-rays”
“I am reminded of the recent Stanford MIRAGE study which found that LLMs will happily hallucinate answers about medical images if the medical images are omitted.”