Uncovering bias and variability in how large language models attribute cardiovascular risk

Area of Science:

• Medical Artificial Intelligence
• Clinical Decision Support Systems
• Health Equity Research

Background:

• Large language models (LLMs) are increasingly integrated into medical applications.
• However, their performance in attributing cardiovascular risk, particularly concerning demographic and clinical factors, is not well understood.
• This gap necessitates an examination of LLM decision-making processes in this critical area.

Purpose of the Study:

• To investigate how a specific LLM (ChatGPT 4.0 mini) assigns relative cardiovascular risk across various demographic and clinical domains.
• To assess the LLM's consistency and potential biases in risk attribution.
• To explore the impact of specific clinical factors (e.g., BMI, diabetes, depression, smoking, hyperlipidemia) on the LLM's risk assessments.

Main Methods:

• A structured set of prompts was designed covering six domains: general cardiovascular risk, BMI, diabetes, depression, smoking, and hyperlipidemia.
• Prompts were submitted in triplicate to ChatGPT 4.0 mini.
• Neutral prompts assessed baseline risk attribution, while comparative prompts evaluated changes in risk assignment when specific domains were included.

Main Results:

• The LLM generally attributed higher cardiovascular risk to men than women and to Black patients compared to white patients across neutral prompts.
• In comparative analyses, sex-based risk attributions shifted in two of six domains (e.g., with depression, risk was equal; with smoking, males were higher risk).
• Race-based risk attributions remained consistent, with the LLM consistently identifying Black patients as higher risk. High agreement across repeated runs (ICC=0.949) was observed.

Conclusions:

• The LLM demonstrated both bias and variability in its cardiovascular risk attributions across different domains.
• While sex-based risk perceptions could be influenced by comorbidities, race-based perceptions were notably stable.
• These findings underscore the critical need for rigorous evaluation of LLMs in clinical settings to mitigate the risk of perpetuating existing health inequities.