Utility of RAND/UCLA appropriateness method in validating multiple-choice questions on ECG

Area of Science:

• Medical Education
• Cardiology
• Health Professions Education

Background:

• Expert consensus is crucial for developing high-quality medical assessment tools.
• Validating multiple-choice questions (MCQs) for electrocardiogram (ECG) interpretation requires rigorous methodology.
• The RAND/UCLA appropriateness method (RAM) offers a structured framework for expert consensus.

Purpose of the Study:

• To evaluate the utility of the RAND/UCLA appropriateness method (RAM) for validating expert consensus-based MCQs on ECGs.
• To assess the effectiveness of RAM in ensuring the quality and appropriateness of ECG assessment questions.
• To determine if RAM can enhance the reliability of MCQ-based evaluations in cardiology.

Main Methods:

• A three-phase RAND/UCLA appropriateness method (RAM) process was employed with nine expert panelists.
• Experts participated in preparatory, online (Round 1), and face-to-face (Round 2) consensus-building phases.
• 100 ECG MCQs were evaluated using median scores and interquartile range (IQR) for appropriateness, followed by expert panel review.

Main Results:

• The average expert success rate in Round 1 was 0.89.
• 54 out of 100 questions were initially categorized as 'Discussion' based on median score and IQR.
• In Round 2, 23% of the original 100 ECG MCQs were ultimately deemed inappropriate, highlighting the method's filtering capability.

Conclusions:

• The RAM, utilizing median score and IQR, provides an objective approach to determining MCQ validity.
• This method aids in selecting appropriate questions, thereby contributing to the development of higher-quality ECG assessment tests.
• RAM enhances the rigor of expert consensus in medical education, particularly for specialized fields like cardiology.