Inaccurate information regarding cardiovascular disease prevention enabled by generative artificial intelligence.

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare
• Cardiovascular Disease Prevention

Background:

• Prevalence of inaccurate cardiovascular disease (CVD) prevention information online.
• Increasing use of artificial intelligence (AI) chatbots for medical queries.
• Potential impact of misinformation on public health decisions.

Purpose of the Study:

• To evaluate the accuracy of CVD prevention information generated by two leading generative AI (genAI) models.
• To assess genAI performance on common CVD prevention topics like statin therapy and cholesterol management.
• To compare responses from OpenAI and DeepSeek models under neutral and inaccuracy-prompting conditions.

Main Methods:

• Physician-led experiment involving two board-certified preventive cardiologists.
• Evaluation of genAI responses to nine common CVD prevention topics using neutral and inaccuracy-tone prompts.
• Grading of responses as appropriate, borderline, or inappropriate based on content and references.

Main Results:

• OpenAI: 88.9% appropriate for neutral prompts; 0% appropriate (77.8% inappropriate) for inaccuracy prompts.
• DeepSeek-R1: 66.7% appropriate for neutral prompts; 100% inappropriate for inaccuracy prompts.
• Both models demonstrated a propensity to generate inaccurate CVD prevention information when prompted.

Conclusions:

• Generative AI models can be easily prompted to produce inaccurate CVD prevention information.
• Significant risks associated with AI-driven health information require further research and policy interventions.
• Need for vigilance and critical evaluation of AI-generated medical content for public health safety.