Internet-based measurement of visual assessment skill of trainee radiologists: developing a sensitive tool

Expert radiologists exhibit high levels of visual diagnostic accuracy from review of radiological images, doing so after accumulating years of training and experience. To train new radiologists, learning interventions must focus on the development of these skills. By developing a web-based measure of image assessment, a key part of visual diagnosis, we aimed to capture differences in the performance of expert, trainee and non-radiologists.

12 consultant paediatric radiologists, 12 radiology registrars, and 39 medical students were recruited to the study. All participants completed a two-part, online task requiring them to visually assess 30 images (25 containing an abnormality) drawn from a library of 150 paediatric skeletal radiographs assessed prior to the study. Participants first identified whether an image contained an abnormality, and then clicked within the image to mark its location. Performance measures of identification accuracy, localisation precision, and task time were collected.

Despite the difficulties of web-based testing, large differences in performance, both in terms of the accuracy of abnormality identification and in the precision of abnormality localisation were found between groups, with consultant radiologists the most accurate both at identifying images containing abnormalities (p < 0.001) and at localising abnormalities on the images (p < 0.001).

Our data demonstrate that an online measurement of radiological skill is sufficiently sensitive to detect group level changes in performance consistent with the development of expertise.

The developed tool will allow future studies assessing the impact of different training strategies on cognitive performance and diagnostic accuracy.