The Predictive Validity of the ABFM's In-Training Examination

Area of Science:

• Medical Education
• Family Medicine

Background:

• The American Board of Family Medicine (ABFM) In-Training Examination (ITE) is a key assessment tool.
• Predictive validity of the ITE for certification exam success is crucial for program evaluation.

Purpose of the Study:

• To assess the predictive validity of the ABFM In-Training Examination (ITE) for future ABFM certification examination performance.
• To analyze score changes across residency levels and evaluate the diagnostic accuracy of PGY3 ITE scores.

Main Methods:

• A repeated measures design was employed across three training levels (PGY1-PGY2, PGY2-PGY3, PGY3-certification).
• Three distinct cohorts from 2010-2013 were analyzed.
• Correlation, score growth, and predictive values (sensitivity, specificity, PPV, NPV) were calculated.

Main Results:

• ITE scores demonstrated a correlation of approximately 0.7 with subsequent ITE or certification exam scores.
• Mean score increases were observed: 52 points (PGY1-PGY2), 34 points (PGY2-PGY3), and 27 points (PGY3-certification).
• Sensitivity, specificity, positive predictive value, and negative predictive value for PGY3 scores predicting certification were 0.91, 0.47, 0.96, and 0.27, respectively.

Conclusions:

• The ABFM In-Training Examination (ITE) is a valuable predictor of resident performance on future ITEs and the certification examination.
• ITE scores provide meaningful insights into a resident's readiness for board certification.