Cost, Usability, Credibility, Fairness, Accountability, Transparency, and Explainability Framework for Safe and...

Area of Science:

• Medical Education Technology
• Artificial Intelligence in Healthcare
• Large Language Models (LLMs)

Background:

• Increasing adoption of LLMs in healthcare presents opportunities for improved accessibility and efficiency.
• Lack of established guidelines for developing LLMs specifically for medical education.
• Need for a structured approach to identify and prioritize LLM development enablers in this domain.

Purpose of the Study:

• To identify and prioritize critical enablers for successful LLM development in medical education.
• To analyze the interrelationships and hierarchical structure among these identified enablers.
• To develop a practical framework for evaluating LLMs in medical education.

Main Methods:

• Narrative literature review to identify initial LLM development enablers.
• Analytical Hierarchy Process (AHP) to determine user-prioritized enabler importance.
• Total Interpretive Structural Modeling (TISM) to analyze developer perspectives on enabler hierarchy.
• MICMAC analysis to assess the driving and dependence powers of enablers.
• Focus groups with a nonprobabilistic purposive sampling approach.

Main Results:

• Credibility emerged as the most critical enabler (priority weight 0.37), followed by accountability (0.276) and fairness (0.106).
• Usability was found to be of negligible importance (0.04) by users.
• Product developers identified cost as the least important enabler, though MICMAC analysis showed its strong influence on other factors.
• TISM results generally aligned with AHP findings, confirming user preferences.
• Focus group inputs were reliable, with a consistency ratio of 0.084.

Conclusions:

• This study provides the first comprehensive analysis of LLM enablers for medical education.
• A prescriptive framework, CUC-FATE (Cost, Usability, Credibility, Fairness, Accountability, Transparency, Explainability), was developed for evaluating LLM enablers.
• Findings offer valuable guidance for healthcare professionals, technology experts, regulators, and policymakers involved in LLM implementation in medical education.