Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
To evaluate clinical AI in realistic settings, the authors tested a specialized clinical agent (Doctorina), four frontier LLMs, and eight practicing physicians across 150 synthetic Polish primary-care consultations requiring dynamic information gathering. Doctorina significantly outperformed human physicians in Top-1 diagnostic concordance (82.0% vs. 57.0%, a 25.0 percentage point margin) while achieving superior normalized scores for diagnostic workup (89.4 vs. 66.9) and treatment planning (83.7 vs. 61.2). Replicated across runs, the results reveal that specialized clinical architectures—followed closely by models like Kimi K3 and Claude Opus—surpass general practitioners across end-to-end adaptive clinical workflows.
Human physicians trail specialized clinical AI by 25 percentage points in diagnostic accuracy when evaluated on realistic, multi-turn primary care consultations rather than static benchmarks.
Clinical AI evaluation should encompass diagnosis and management after adaptive information gathering. We compared Doctorina, eight physicians and four standalone frontier language models in 150 synthetic Polish-language primary-care consultations. Doctorina achieved 82.0% Top-1 concordance versus 57.0% for physicians (difference, 25.0 percentage points; 95% confidence interval, 17.7-32.7) and 97.3% versus 85.0% primary-or-reference-differential concordance. Across 149 case pairs, normalized workup and treatment scores were 89.4 versus 66.9 and 83.7 versus 61.2. Doctorina had the highest diagnostic point estimates among all six groups; Kimi K3 ranked next, while Claude Opus 5 led the closely spaced management estimates of Opus, Doctorina and Kimi. A second Doctorina execution reproduced the advantages over physicians across all outcomes. Doctorina's advantage over physicians therefore extended from primary-diagnosis selection to higher-rated diagnostic workup and initial treatment after adaptive consultation.