Understanding Clinical Reasoning Variability in Medical Large Language Models: A Mechanistic Interpretability Study

Area of Science:

• Medical Artificial Intelligence
• Clinical Decision Support Systems
• Computational Linguistics in Medicine

Background:

• Medical large language models (LLMs) demonstrate high benchmark accuracy but exhibit unexplained clinical reasoning variability and errors.
• Sparse autoencoders offer a mechanistic interpretability approach to understand LLM knowledge representation and failure modes in medicine.
• Existing benchmarks may not capture the full spectrum of clinical reasoning stability required for safe AI deployment.

Purpose of the Study:

• To evaluate the clinical reasoning stability of distinct medical LLM architectures (GPT-5, MedGemma-27B-Text-IT, OpenBioLLM-Llama3-70B) under systematic perturbations.
• To analyze how different model architectures encode polysemous medical terms using sparse autoencoders and ablation experiments.
• To assess the efficacy of a retrieval intervention for disambiguating medical term senses and improving model performance.

Main Methods:

• Evaluated reasoning stability using 355 systematic perturbations in oncology cases, comparing staging and treatment against NCCN and AJCC guidelines.
• Trained sparse autoencoders on 1 billion tokens from MIMIC-IV clinical notes to analyze encoding of polysemous medical terms.
• Conducted 850 ablation experiments and tested a two-stage retrieval intervention for sense disambiguation.

Main Results:

• Models showed dramatic reasoning instability; OpenBioLLM staging accuracy varied from 45.9% to 99.1% based on prompt format.
• Sparse autoencoder analysis revealed significant differences in encoding: MedGemma showed 77.8% feature overlap across word senses, OpenBioLLM 13.6%.
• A retrieval intervention improved MedGemma disambiguation by 10.2% but harmed OpenBioLLM by 2.0%, indicating architecture-specific intervention effects.

Conclusions:

• Medical AI systems exhibit clinical reasoning fragility not captured by benchmark performance, highlighting the need for deeper interpretability.
• Architecturally distinct models encode medical concepts differently, meaning interventions effective for one may harm another.
• Safety validation for medical AI must be architecture-specific, as benchmark equivalence does not guarantee functional equivalence.