Pilot Implementation and Psychometric Analysis of a Multisource Feedback Instrument for Surgical Resident Leadership...

Area of Science:

• Medical Education
• Surgical Residency Training
• Leadership Assessment

Background:

• The Inpatient Leadership Assessment Device (I-LEAD) was developed to evaluate essential leadership skills for surgical residents leading inpatient teams.
• Multisource feedback (MSF) is crucial for assessing complex skills like leadership in a clinical setting.

Purpose of the Study:

• To evaluate the feasibility of using the I-LEAD instrument in real-world clinical environments.
• To gather psychometric evidence supporting the validity and reliability of I-LEAD leadership scores.

Main Methods:

• A 27-item online survey (I-LEAD) was administered to surgical attendings, advanced practice providers (APPs), interns, and senior residents.
• A many-facet Rasch model (MFRM) was used to analyze psychometric properties, including resident leadership ability, item difficulty, and rater severity.
• Data were collected from a general surgery residency program over nine months, encompassing six inpatient services.

Main Results:

• The I-LEAD instrument demonstrated feasibility in collecting structured MSF, with a mean rating of 3.71 out of 4.
• All 27 items fit the MFRM, indicating good construct validity.
• Statistically significant differences in scores were observed across rater roles (p < 0.001), with attendings and interns rating higher than APPs and senior residents, highlighting the value of diverse perspectives.

Conclusions:

• The I-LEAD instrument is a feasible method for obtaining structured MSF on senior surgical resident leadership behaviors in inpatient settings.
• Differences in ratings across rater roles underscore the importance of incorporating varied team member perspectives for comprehensive leadership assessment.
• Future research should focus on developing and evaluating educational interventions to improve resident leadership skills.