ChatGPT in Public Cardiovascular Healthcare: Accuracy, Limitations, and Implications

Area of Science:

• Artificial Intelligence in Healthcare
• Cardiovascular Medicine
• Clinical Decision Support

Background:

• Artificial intelligence (AI) tools like ChatGPT are increasingly used in healthcare.
• Research on ChatGPT's effectiveness in public healthcare for cardiac patients is limited.
• This study evaluates ChatGPT for managing cardiovascular health in patients with acute or chronic cardiac conditions.

Purpose of the Study:

• To assess ChatGPT's diagnostic accuracy for cardiac conditions.
• To evaluate ChatGPT's recommendations for supplementary exams and laboratory tests.
• To compare ChatGPT's performance against physician records and other AI models.

Main Methods:

• Analysis of real medical records from a university outpatient clinic's 'Cardiovascular Care' program.
• Evaluation of ChatGPT's ability to analyze cases, propose diagnoses, and recommend actions.
• Assessment of accuracy and errors based on disease severity, rarity, mortality risk, and urgency.

Main Results:

• ChatGPT achieved 43% accuracy in diagnostic hypotheses but only 5% for exams and 10% for lab tests.
• It demonstrated discernment in diagnosing based on condition factors but not in recommending tests.
• ChatGPT showed the highest diagnostic accuracy among evaluated AI models, yet still produced incorrect outputs.

Conclusions:

• ChatGPT exhibits some diagnostic capabilities but its overall reliability for clinical decision-making is questionable.
• Performance can approach random chance, necessitating caution.
• Further research is needed to determine its safe and effective integration into cardiac patient management.