Expert panel consensus on assessment checklists for a rheumatology objective structured clinical examination

Area of Science:

• Medical Education
• Rheumatology Assessment

Background:

• Regional rheumatology fellowship groups utilize objective structured clinical examinations (ROSCEs).
• Existing ROSCEs lack validation for cross-programmatic use.
• Standardized assessment tools are crucial for consistent medical training.

Purpose of the Study:

• To achieve expert agreement on checklist items for multiple ROSCE stations.
• To develop a validated framework for rheumatology objective structured clinical examinations.
• To enhance the reliability and consistency of rheumatology fellowship assessments.

Main Methods:

• A single-round survey was distributed to 127 US rheumatology educators.
• 173 potential checklist items for 11 ROSCE stations were evaluated.
• Consensus was defined as a lower 95% confidence interval bound ≥4.0 on a 5-point Likert scale.

Main Results:

• 25 educators completed the survey, representing a 20% response rate.
• 133 out of 173 items (77%) achieved consensus for retention.
• Item retention varied by station, with patient counseling and radiograph interpretation at 100%, and rehabilitation medicine at 24%.

Conclusions:

• Expert consensus was reached on 133 items for 11 proposed ROSCE stations.
• The survey methodology allows for diverse geographic representation and rigorous statistical validation.
• This validated approach to checklist content agreement is applicable across various medical disciplines.