“To me, this is the best kind of quantified, reasoned basis to advance use of a diagnostic AI adjunct. It's been seen to perform. I would hope they continue to independently review performance against diagnosticians, and maintain a dataset of performance over time. It's by no means a given it continuously improves unless the model acquires matching compet…”
Teaching large language models to reason like expert diagnosticians
Differential diagnosis is an iterative process that integrates patient information with broader medical knowledge. Clinical case series such as the NEJM Clinicopathologic Conferences (CPCs), published continuously since 1923, feature expert physicians who demonstrate diagnostic reasoning to peers, and have been used for decades to evaluate AI. However, prior AI evaluations have largely focused on final diagnostic accuracy rather than nuanced clinical reasoning. Here, we introduce Dr. CaBot, an agentic AI system that emulates an expert diagnostician by generating written and narrated slide-based presentations from an initial case description alone. CaBot recently generated the first AI diagnosis published in the 100+ year history of the NEJM CPCs. In blinded evaluations, physicians misclassified the source of the differential (CaBot vs. physician-written) in 46/62 (74%) of trials and rated them favorably across quality dimensions. When tasked with solving cases for 72 patients with undiagnosed disease from the NIH Undiagnosed Diseases Network, CaBot identified the working diagnosis in 50/72 (69%) of cases from referral notes alone. To promote transparency and research, we also developed CPC-Bench, a physician-validated benchmark based on 7,102 CPCs and 47,648 questions across 10 tasks. We show that CaBot outperforms frontier models on CPC-Bench, and release both CaBot and CPC-Bench publicly to foster progress in clinical AI.
候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。
HN 书评
没有评论直接链到它;下面是它被提交的讨论串里回复最多的评论(不计入统计)“This was using o3. GPT 5.2/5.3 should be much improved. Just like software engineering, it may be best to leave it up to the AI to do the work but let a human guide it and check it.”
- Advancing medical artificial intelligence using a century of cases ↗hhs · 2025-10-10 · 3 分
- LLMs exceed physicians on complex text-based differential diagnosis ↗rippeltippel · 2026-02-13 · 2 分