Interdisciplinary Development and Fine-Tuning of CARDIO, a Large Language Model for Cardiovascular Health Education...

Area of Science:

• Artificial Intelligence in Healthcare
• Public Health
• Computer Science

Background:

• Chronic illnesses affect millions, with significant impacts on healthcare costs and patient well-being.
• People living with HIV often face challenges managing cardiovascular and metabolic comorbidities.
• Effective patient education is crucial for managing chronic conditions and improving health outcomes.

Purpose of the Study:

• To describe the development and evaluation of an intersectionality-informed large language model (LLM) for patient education.
• To optimize health through prevention education for cardiovascular and metabolic comorbidities in persons living with HIV.
• To present a tutorial for interdisciplinary development of a specialized LLM (CARDIO) for cardiovascular health in HIV care.

Main Methods:

• Curated a comprehensive dataset from authoritative sources and public HIV forums.
• Benchmarked candidate LLMs and fine-tuned a LLaMA-based model using GPT-4 and reinforcement learning.
• Employed iterative refinement with quantitative metrics and qualitative expert evaluations.

Main Results:

• Pre-existing LLMs showed poor accuracy, readability, and professionalism.
• The fine-tuned CARDIO model demonstrated significantly improved performance (Accuracy 5.0, Readability 4.98, Professionalism 4.98).
• CARDIO achieved better readability (Kincaid 7.17) and reduced jargon (2.92) compared to baseline models.

Conclusions:

• A customized, data-driven LLM like CARDIO shows potential for personalized, culturally relevant patient education.
• Targeted data curation, rigorous benchmarking, and iterative fine-tuning are key to developing effective AI health tools.
• This work establishes a foundation for AI-driven strategies in managing comorbid conditions within the HIV population.