Performance on PMPs and performance in practice: are they related?

Area of Science:

• Medical Education
• Clinical Skills Assessment
• Healthcare Simulation

Background:

• Assessing clinical skills is crucial for patient safety.
• Patient management problems (PMPs) are a common tool for evaluating medical competence.
• The validity of PMPs in reflecting real-world performance needs rigorous testing.

Purpose of the Study:

• To test the criterion validity of patient management problems (PMPs).
• To compare performance on PMPs with performance in actual clinical practice settings.

Main Methods:

• A direct behavior-by-behavior comparison was used.
• PMP scenarios were replicated in a clinical setting using role-playing actors.
• Participants believed the actors were real patients during the encounters.

Main Results:

• Significant inconsistencies were found between PMP performance and practice performance.
• Substantially greater errors of omission and commission were observed in the practice setting compared to PMPs.
• This suggests PMPs may not accurately predict real-world clinical errors.

Conclusions:

• The criterion validity of the tested PMPs is questionable.
• Current PMP assessments may not adequately prepare clinicians for the complexities of real patient care.
• Further research is needed to develop more valid assessment tools for clinical practice.