Evaluating GPT-5 as a generalist multimodal clinical model

The transition from task-specific artificial intelligence toward general-purpose foundation models raises fundamental questions about their capacity to support the integrated reasoning required in clinical medicine, where diagnosis demands synthesis of ambiguous patient narratives, laboratory data, and multimodal imaging.

We provide the first controlled, cross-sectional evaluation of the GPT-5 family (GPT-5, GPT-5 Mini, and GPT-5nano) against its predecessor GPT-4o across a diverse spectrum of clinically grounded tasks, grounded tasks, including the United States Medical Licensing Examination, text-based, text-based reasoning benchmarks (MedQA and MedXpertQA Text), and visual question answering (VQA) in neuroradiology (BraTS), digital pathology (PathVQA and BreakHis), and mammography (EMBED, InBreast, Chinese Mammography Database, and CBIS-DDSM) using a standardized zero-shot chain-of-thought protocol.

GPT-5 demonstrated substantial performance gains with absolute improvements exceeding 25 percentage points on MedXpertQA. When tasked with multimodal synthesis, GPT-5 effectively leveraged its enhanced training corpora to achieve state-of-the-art or competitive performance across most VQA benchmarks and outperformed GPT-4o by margins of 10% to 40% in mammography tasks requiring fine-grained lesion characterization. However, performance remained moderate in neuroradiology (44% macro-average accuracy) and lagged substantially behind domain-specific models in mammography, where specialized systems exceed 80% accuracy compared with GPT-5's 52% to 64%.

These findings indicate that although GPT-5 represents a meaningful advance toward general, multimodal clinical inference, generalist models are still not yet substitutes for purpose-built systems in highly specialized, perception-critical tasks. We delineate the current VQA landscape, positioning GPT-5 as a powerful adjunct capable of holistic inference while underscoring that domain adaptation, rigorous validation, and guarantees of reasoning transparency remain essential prerequisites for clinical deployment.