Microsoft’s diagnostic orchestrator, MAI-DxO, now has a peer-reviewed paper in Science. The headline numbers are striking; the test conditions are what decide how much they mean.
The results
- 304 complex New England Journal of Medicine cases: MAI-DxO, running on OpenAI’s GPT-5, reached the correct diagnosis in 83.2% of cases. Generalist doctors with access to textbooks and the internet scored 36.1%.
- 325 emergency department admissions: 80.3% for MAI-DxO versus 46.8% for the human group.
- How it works: the system steps through a case, decides which questions to ask and tests to order, and simulates a panel of clinical experts.
- Cost: researchers say its work-ups cost less, but those are estimates from the cases, not real billing.
Where to be skeptical
- Text only. Neither side had physical exams or imaging. Real diagnosis leans on both.
- Weak comparator. The doctors were generalists, not specialists, working on unusually hard cases. A 36% baseline reflects that setup, not routine practice.
- Published puzzle cases. NEJM cases are curated; model exposure to similar material is a standing concern with benchmarks like this.
- Not deployed. It is not used in clinical practice. Microsoft’s researchers plan to work with Beth Israel Deaconess Medical Center to test emergency-department triage, which is the evidence that will matter.
Takeaway
Peer review lifts this above a press-release claim, and the orchestration approach (structured questioning plus cost awareness) is the interesting part. But it is a strong benchmark result, not evidence of better patient outcomes. Watch the prospective trial, not the percentage.
This is general information, not medical advice.
