Validation of a method for assessing resident physicians' quality improvement proposals

Area of Science:

• Medical Education
• Healthcare Quality Improvement
• Research Methodology

Background:

• Residency programs require trainees to engage in quality improvement (QI) projects.
• Assessing the competency of trainees in systems-based practice and practice-based learning through QI projects is crucial.
• A lack of validated instruments for assessing QI proposals hinders effective evaluation.

Purpose of the Study:

• To develop and validate an instrument for assessing resident QI proposals.
• To determine the reliability and validity of the Quality Improvement Proposal Assessment Tool (QIPAT-7).

Main Methods:

• Content for QIPAT-7 was derived from a national panel of QI experts.
• The instrument underwent iterative refinement, pilot testing, and revision.
• Seven raters assessed 45 resident QI proposals using QIPAT-7, with principal factor analysis, Cronbach's alpha, and intraclass correlations used for validation.

Main Results:

• Principal factor analysis indicated that QIPAT-7 items form a single dimension.
• High interrater reliability was observed for each item (range 0.79–0.93).
• High internal consistency reliability was found among the items (Cronbach's alpha = 0.87).

Conclusions:

• The QIPAT-7 demonstrates content and internal structure validity for assessing resident physician QI proposals.
• QIPAT-7 is a reliable and useful tool for evaluating resident QI proposals.
• Future research should explore QIPAT-7 reliability across diverse training programs and correlate scores with QI success metrics.