Validity evidence for a novel multiple-choice question bank in the field of child abuse pediatrics

Though child abuse pediatrics has been a board-certified subspecialty for 15 years, there are few formalized board preparation resources available.

The purpose of this project was to establish a multiple-choice question bank with sufficient validity evidence for use in preparation for the child abuse pediatrics board examination.

The question bank was distributed via an electronic child abuse pediatrics mailing list. Participants completing the entire question bank included 27 board-certified child abuse pediatricians (CAPs), 19 board-eligible CAPs, and 18 CAP fellows.

We used Messick's framework to conduct the validity investigation, which includes five components: content evidence, response process, internal structure, relation to other variables, and consequences. Item analyses included difficulty index, discrimination index, and distractor analysis. We used Cronbach's alpha to estimate internal consistency reliability. We conducted linear regressions of scores on the question bank compared to in-training exam scores and career stage.

Eighty-four participants completed part of the question bank, and 64 completed the entire question bank. Of the original 117 questions ("items"), 94 met inclusion criteria. The mean score among board-certified CAPs was 80 %, and among participants reporting passing third-year ITE scores was 81 %. Correlation coefficient of scores on this question bank by career stage was r = 0.94, and by year of fellowship was r = 0.99. Cronbach's alpha for internal consistency reliability was 0.83.

This multiple-choice question bank is the first question bank with a robust validity investigation for use by child abuse pediatrics trainees.