Microsoft AI Panel Outperforms Physicians in Complex Diagnostics

Microsoft has unveiled a new diagnostic AI system that significantly outperformed human physicians in a rigorous simulation of complex medical cases. The results, published in Science this week, indicate that the company’s multi-agent framework, known as MAI-DxO, correctly identified diagnoses in 83% of difficult scenarios. In contrast, primary care doctors and general internists, who worked individually with reference materials, achieved a 36% success rate. Mustafa Suleyman, CEO of Microsoft AI, described the achievement as a critical step toward achieving "medical superintelligence." This latest development marks a significant milestone in the race among major technology firms to demonstrate that artificial intelligence can match or exceed human medical expertise.

The system operates by coordinating multiple AI agents that function similarly to a panel of physicians. These agents interact with the patient, order necessary tests, and debate potential diagnoses before reaching a conclusion. The study utilized complex cases derived from the New England Journal of Medicine to test these capabilities. It is important to note that these specific types of cases are typically managed by specialists or team-based approaches in real-world settings, meaning the comparison was against generalist physicians rather than the specialists who would usually handle such difficult diagnoses.

Dominic King, Vice President of Health at Microsoft AI, emphasized the significance of the findings in addressing current healthcare challenges. He stated that the primary issue in medical delivery is the growing mismatch between the demand for and supply of medical expertise. The study’s data suggests that much of the performance improvement stems from the use of more powerful underlying AI models, with Microsoft’s orchestration system providing additional gains. While the system was tested with models from various providers, including OpenAI, Google, Anthropic, Meta, and DeepSeek, the highest results were achieved using OpenAI’s GPT-5 and GPT-5.5 models.

Cost and accuracy were both measured in the evaluation. When using GPT-5 with a basic prompt, the system scored 78.9% at an estimated cost of $8,400 per case. The MAI-DxO orchestration reduced this cost by approximately $3,000 while improving accuracy by about four points. However, for emergency department cases, the addition of the orchestration layer offered little benefit in terms of either cost or accuracy. With the newer GPT-5.5 model, the system improved accuracy on complex cases from 72% to nearly 87%, but again saw minimal gains in emergency settings, where both versions scored around 85%.

Looking ahead, Microsoft is collaborating with the Mayo Clinic to develop a foundation model specifically for healthcare, with a release planned for next year. Suleyman believes this model will enable significant leaps in the field. Additionally, a new partnership with Beth Israel Deaconess Medical Center in Boston will allow for real-world evaluations of the system’s clinical reasoning across various hospital settings. This study will compare the AI’s recommendations against those of human doctors in both emergency and urgent care environments. The initiative coincides with a broader manifesto from Microsoft outlining the requirements for medical superintelligence, aiming to integrate AI into the full arc of patient care rather than just answering isolated medical questions.

Suleyman, who previously co-founded DeepMind, expressed confidence that healthcare models will improve by an order of magnitude within the next two to three years. He anticipates that by 2027, the company will be close to a breakthrough in coding medical records, allowing for highly accurate predictions of patient outcomes. This foresight, he noted, is invaluable to clinicians. Microsoft views this work as an expansion of its existing tools for dictation and radiology, aiming to hold the top position in clinical decision support products in the coming years.