Pilot Implementation and Psychometric Analysis of a Multisource Feedback Instrument for Surgical Resident Leadership...

Previously, we developed the Inpatient Leadership Assessment Device (I-LEAD), a multisource feedback (MSF) instrument assessing the essential leadership skills for surgical resident physicians leading inpatient teams. This pilot study sought to evaluate the feasibility of utilizing I-LEAD in real-world contexts and acquire psychometric evidence to support the validity and reliability of its leadership scores.

At the end of a clinical rotation, surgical attendings, advanced practice providers (APPs), interns, and senior residents were invited to complete the I-LEAD instrument, which is structured as a 27-item online survey. Ratings were summarized with descriptive statistics, and psychometric properties were characterized with a many-facet Rasch model (MFRM) encompassing measures of resident leadership ability, item difficulty, and rater role severity.

An academic general surgery residency program at a tertiary hospital in the Northeast United States over 9 months during the 2024 to 2025 academic year. It encompassed 6 inpatient general surgery services.

Twenty senior residents were evaluated by 8 attendings, 7 APPs, and 19 interns.

Residents had a mean of 3.1 evaluations. Overall, 65.5% of all item ratings were "Excellent," 24.0% "Good," 2.5% "Marginal," 0.3% "Poor," and 7.6% "Unable to assess," resulting in a mean of 3.71 of 4. All 27 items demonstrated good fit with the MFRM. There was also good fit among rater roles with statistically significant differences in scores across groups (p < 0.001): attendings and interns assigned higher scores than APPs and senior residents. The model demonstrated strong reliability.

It is feasible to solicit structured MSF from surgical inpatient team members regarding senior resident leadership behaviors via the I-LEAD instrument. Differences in scores across rater roles emphasizes the value of soliciting input from team members with varying perspectives. Next steps are to create and assess the effectiveness of educational interventions to enhance specific resident leadership behaviors.