Agreement between trained and untrained observers in grading intravascular bubble signals in ultrasonic images

In our experience, it is easier to identify gas bubbles in ultrasonic images than in aural Doppler signals. To verify this, we asked 27 observers with no previous training to estimate the quantity of gas bubbles in video tapes containing sequences of ultrasound images recorded during decompression experiments. The amount of bubbles was graded according to a non-linear grading system with six levels. The results obtained were compared to evaluations performed on-site by a trained observer. Approximately 70% of the evaluations performed by the untrained observers agreed completely with the on-site gradings, and more than 95% agreed within 1 grade unit. The strength of agreement can be described by use of the weighted kappa statistic, and we have compared the agreement in our study with agreement obtained in a previous study using Doppler signals for bubble detection. We find that in grading bubble signals in images, untrained observers perform equally as well as trained observers grading bubbles in Doppler signals. We conclude that ultrasonic imaging offers a useful and cost-effective alternative to Doppler systems for detection and quantification of intravascular gas bubbles.