Sex-based diagnostic bias in AI symptom checkers: a structured, exploratory audit of three platforms

Artificial intelligence symptom checkers are used by hundreds of millions of people annually as first-line triage tools, and their outputs may influence whether and how quickly a person seeks emergency care. We conducted a structured, exploratory audit of three widely used platforms, Ada Health, Isabel Healthcare, and Docus AI, across four standardized clinical vignettes [acute coronary syndrome (ACS) with chest pressure, ACS without chest pressure, pulmonary embolism, and panic attack], entering each vignette for matched male and female synthetic patient profiles with sex as the sole variable altered. We observed sex-based divergence in outputs from all three platforms, appearing as differences in differential diagnosis rank, differences in assigned probability, and differences in generated explanatory or advisory text. The direction of this divergence was not uniform: some findings were unfavorable to the female profile and others were favorable to it. Isabel Healthcare ranked Heart Attack first for the male profile but third for the female profile on the ACS with chest pressure vignette, and ranked Heart Attack seventh rather than fifth for the female profile on the panic attack vignette; on the ACS without chest pressure vignette, the same platform ranked Heart Attack first for the female profile, ahead of the male profile. Docus AI assigned a lower ACS probability to the female profile on the ACS without chest pressure vignette while its own text described this presentation as common in women, and applied a mirrored justification, common in males, to support a higher probability for the male profile given identical symptoms; the platform's closing advisory text also described immediate medical attention as crucial for the male profile and prompt evaluation as merely important for the female profile. Ada Health's explanatory text for the female profile stated that heart attacks are slightly more common in men than women, presented alongside a demoted cardiac ranking. These findings indicate that AI symptom checkers can produce sex-based divergence in their outputs, including an internally inconsistent, self-contradictory justification on one platform, and support the case for equity auditing of such systems, with attention to both numerical outputs and generated explanatory text, before and after deployment.