Scoring patient management problems: external validation of expert consensus

Area of Science:

• Medical Education
• Clinical Assessment

Background:

• Assessing physician competence in patient management is crucial.
• Standardized methods are needed for evaluating clinical decision-making skills.

Purpose of the Study:

• To evaluate agreement among medical faculty in rating diabetes mellitus patient management problems.
• To determine if expert consensus ratings predict well-qualified physicians' choices.

Main Methods:

• Two written patient management problems in diabetes mellitus were used.
• Medical faculty rated problem options.
• Consensus ratings were correlated with endocrinology fellows' decisions.

Main Results:

• Faculty agreement varied from 31% to 73% across problem components.
• Consensus ratings strongly predicted endocrinology fellows' choices (correlations .60–.97).

Conclusions:

• Expert consensus ratings demonstrate high concurrent validity for well-qualified clinicians.
• Scoring weights based on expert consensus can reliably assess clinical decision-making.