“I think it only proves very little, because a typical conversation they studied looks like this:”
Large Language Models Pass the Turing Test
三方图灵测试中 GPT-4.5 有 73% 的时间被判为真人
作者在两个独立人群上做了随机对照、预先注册的三方图灵测试,评估 ELIZA、GPT-4o、LLaMa-3.1-405B 和 GPT-4.5:参与者同时与一名真人和其中一个系统各聊 5 分钟,再判断谁是人类。被提示扮演类人角色时,GPT-4.5 有 73% 的时间被判为人类,显著高于真人被选中的比例;LLaMa-3.1 为 56%,与真人无显著差异;基线 ELIZA 和 GPT-4o 只有 23% 和 21%,显著低于随机水平。作者称这是首个人工系统通过标准三方图灵测试的实证。
评论者拿它说明 LLM 已经通过标准的三方图灵测试,常引用 GPT-4.5 的 73% 并以 ELIZA 作基线对照,也有人批评该测试其实只能证明很少。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“ChatGPT 4.5 was judged to be the human 73% of the time in this RCT study, where human interrogators had 5-minute conversations with a human and an LLM: Large Language Models Pass the Turing Test”
“I only skimmed it, but I don't see anything clearly wrong about it.”
“but some of the most recent models have actually passed this test [1].”
“I think it illustrates both the limits of the test and that LLMs are getting quite good.”
“The latter: Large Language Models Pass the Turing Test”
看全部 20 条 →
“With regards to strong AI, there was a widely accepted definition - i.e. passing the Turing Test - and we are way past that point already: ( see Large Language Models Pass the Turing Test )”
“At this point LLMs usually beat humans at the Turing Test! People are more likely to pick the LLM as the human, rather than the human. Large Language Models Pass the Turing Test”
“That researchers haven't done Turing-test studies (taking the setup from turing and even called them that) is patently false. Including openly testing LLMs:”
“And still had people think it was a real person, over 27% of the time.”
“such a test was run last year and GPT-4.5 and LLaMa-3.1-405B both passed.”
“I had already found this study, published almost a year ago, saying that they do:”
“In the test mentioned in nearby comments ELIZA only got 27% suggesting the test wasn't that easy to fool.”
“73% judged GPT 4o to be the human.”
“By most normal standards, they can definitely pass the Turing test.”
“When prompted to adopt a humanlike persona, GPT-4.5 was judged to be the human 73% of the time”
“From the abstract: "When prompted to adopt a humanlike persona, GPT-4.5 was judged to be the human 73% of the time”
“From the abstract: "When prompted to adopt a humanlike persona, GPT-4.5 was judged to be the human 73% of the time”
“LLMs started meaningfully passing the Turing test a year or two ago, around GPT-4.5.”
“has been beaten by all manner of models.”