Comparison of Artificial Intelligence Models and Human Experts in Managing Dyslipidemia: Assessment of Adherence to...

Objective The objective of this study is to compare guideline adherence between artificial intelligence (AI) models (Claude-3 (Anthropic, San Francisco, CA), DeepSeek-V2 (DeepSeek, Hangzhou, China), GPT-4 (OpenAI, San Francisco, CA)) and human experts in dyslipidemia management using standardized clinical scenarios based on 2019 European Society of Cardiology (ESC)/European Atherosclerosis Society (EAS) and 2021 ESC prevention guidelines. The study employed a comprehensive evaluation framework to capture the holistic nature of dyslipidemia management across multiple interconnected domains. Methods Thirty fictitious but clinically representative cases were developed by lipid specialists across five domains: cardiovascular risk assessment, lipid management, lifestyle modifications, pharmacotherapy, and special populations. This broad scope was deliberately chosen to evaluate the full complexity of integrated cardiovascular risk management as it occurs in clinical practice. Cases included all variables required for objective guideline application. AI models and clinicians (professors, specialists, residents) provided management recommendations. A blinded assessment paradigm was employed to minimize potential evaluation bias, with evaluators scoring responses using alphanumeric coding to prevent source identification bias. Responses were assessed using standardized rubrics (0-3 scales) for four equally-weighted parameters: accuracy (guideline concordance), comprehensiveness (clinical coverage), applicability (implementation feasibility), and efficacy (simulated low-density lipoprotein cholesterol (LDL-C) target attainment). Composite scores were calculated by summing all parameters (maximum 12 points). Results Correct response rates were 91% for AI, 72% for professors, 50% for specialists, and 21-32% for residents. Composite scores (mean ± SD/12) were 10.3 ± 1.0 for AI, 8.1-9.2 for professors, 7.4 ± 1.5 for specialists, and 5.2-6.2 for residents. AI excelled in literal guideline application while professors considered contextual factors (frailty, life expectancy). Professors primarily erred in LDL-C targets (using <100 vs. <55 mg/dL), while AI in nuanced risk stratification. Simulated outcomes showed LDL-C target attainment of 83% with AI, 64% with professors, and 92% with a combined approach. Conclusion AI demonstrated superior guideline adherence in standardized scenarios but may miss contextual clinical factors. The hybrid AI-human approach optimized outcomes, suggesting that augmented intelligence represents the most promising implementation strategy. Limitations include simulated cases (n = 30), potential performance bias favoring literal interpretation, and lack of real-world complexity. Prospective clinical validation is warranted.