Using an Accountability Program to Improve Psychiatry Resident Scores on In-Service Examinations

The Psychiatry Resident-In-Training Examination (PRITE) is a standardized examination that measures residents' educational progress during residency training. It also serves as a moderate-to-strong predictor of later performance on the board certification examination.

This study evaluated the effectiveness of an accountability program used by a public psychiatric hospital to increase its residents' PRITE scores.

A series of consequences and incentives were developed based on levels of PRITE performance. Poor performance resulted in consequences, including additional academic assignments. Higher performance led to residents earning external moonlighting privileges. Standardized PRITE scores for all residents (N = 67) over a 10-year period were collected and analyzed. The PRITE examination consists of 2 subscales-psychiatry and neurology. Change in the overall level of PRITE scores following the implementation of the accountability program was estimated using a discontinuous growth curve model for each subscale.

Standardized scores on the psychiatry subscale were 51.09 points, approximately 0.50 SD change, which was higher after the accountability program was implemented. Standardized scores on the neurology subscale did not change.

An accountability program that assigns consequences based on examination performance may be moderately successful in improving scores on the psychiatry subscale scores of the PRITE. This likely has longer-term benefits for residents due to the relationship between PRITE and board certification examination performance.