Development, Validation, and Implementation of a Medical Judgment Metric

Area of Science:

• Medical Education
• Clinical Skills Assessment
• Healthcare Simulation

Background:

• Medical decision-making is crucial but under-researched in medical training.
• Developing tools to objectively assess clinical judgment is a priority.

Purpose of the Study:

• To create the Medical Judgment Metric (MJM), a rubric for quantifying medical decisions in simulated practice.
• To gather preliminary data on the MJM's reliability and validity.

Main Methods:

• MJM items and domains were derived from established frameworks.
• Content validity was assessed by eight experts.
• The MJM was pilot tested in four simulations with 40 participants and three raters.

Main Results:

• High inter-rater consistency was observed for MJM scores (ICC 0.965–0.987).
• Raters showed strong agreement on expected patient outcomes (Fleiss' Kappa 0.791–0.906).
• MJM scores accurately predicted patient outcomes (Cohen's Kappa 0.851–0.880).

Conclusions:

• The Medical Judgment Metric (MJM) shows initial evidence of reliability and validity.
• The MJM is a promising tool for evaluating medical decision-making in simulated environments.
• Further research is warranted to establish the MJM's utility in medical education.