A construct validity study of clinical competence: a multitrait multimethod matrix approach.

Area of Science:

• Medical Education Research
• Psychometrics
• Healthcare Professional Assessment

Background:

• Assessing clinical competence in high-stakes examinations is crucial for physician practice readiness.
• Existing assessment instruments require robust validation of the constructs they measure.
• Understanding the validity of these assessments ensures reliable evaluation of physicians.

Purpose of the Study:

• To provide evidence for estimating the construct validity of clinical competence.
• To evaluate assessment instruments used in high-stakes physician examinations.
• To determine the practice readiness of physicians through validated assessments.

Main Methods:

• Utilized a multitrait multimethod (MTMM) matrix approach.
• Assessed 39 international physicians using objective structured clinical examinations, in-training evaluation reports, and clinical assessments.
• Examined doctor-patient relationship, clinical competence, and communication skills.

Main Results:

• Reliability of assessments ranged from moderate to high (Cronbach's alpha: 0.58-0.98).
• Convergent and divergent validity were evidenced for clinical competence, doctor-patient relationships, and communication skills (coefficients = 0.12-0.85).
• Method specificity accounted for 23.7% of the variance in assessments.

Conclusions:

• The study provides evidence for the construct validity of all three assessed traits across three methods.
• The multitrait multimethod approach is a valuable, though underutilized, tool for validating complex constructs like clinical competence.
• Findings support the use of validated assessment instruments for determining physician practice readiness.