Evaluation of trainees' ability to perform obstetrical ultrasound using simulation: challenges and opportunities

Evaluation of trainee's ability in obstetrical ultrasound is a time-consuming process, which requires involving patients as volunteers. With the use of obstetrical ultrasound simulators, virtual reality could help in assessing competency and evaluating trainees in this field.

The objective of the study was to test the validity of an obstetrical ultrasound simulator as a tool for evaluating trainees following structured training by comparing scores obtained on obstetrical ultrasound simulator with those obtained on volunteers and by assessing correlations between scores of images and of dexterity given by 2 blinded examiners.

Trainees, taking the 2013 French national examination for the practice of obstetrical ultrasound were asked to obtain standardized ultrasound planes both on volunteer pregnant women and on an obstetrical ultrasound simulator. These planes included measurements of biparietal diameter, abdominal circumference, and femur length as well as reference planes for cardiac 4-chamber and outflow tracts, kidneys, stomach/diaphragm, spine, and face. Images were stored and evaluated subsequently by 2 national examiners who scored each picture according to previously established quality criteria. Dexterity was also evaluated and subjectively scored between 0 and 10. The Raghunathan's modification of Pearson, Filon's z, Spearman's rank correlation, and analysis of variance tests were used to assess correlations between the scores by the 2 examiners and scores of dexterity and also to compare the final scores between the 2 different methods.

We evaluated 29 trainees. The mean dexterity scores in simulation (6.5 ± 2.0) and real examination (5.9 ± 2.3) were comparable (P = .31). Scores with an obstetrical ultrasound simulator were significantly higher than those obtained on volunteers (P = .027). Nevertheless, there was a good correlation between the scores of the 2 examiners judging on simulation (R = 0.888) and on volunteers (R = 0.873) (P = .81).

An obstetrical ultrasound simulator is as good a method as volunteer-based examination for evaluating practical skills in trainees following structured training in obstetrical ultrasound. The threshold for success/failure should, however, be adapted as candidates obtain higher scores on the simulator. Advantages of the obstetrical ultrasound simulator include the absence of location and time constraints without the need to involve volunteers or to interfere with the running of ultrasound clinics. However, an obstetrical ultrasound simulator still lacks the ability to evaluate the trainees' ability to interact with patients.