Subjective differentiation of normal and pathological bronchi on thin-section CT: impact of observer training

The effect of observer training on sensitivity, specificity and interobserver agreement in the differentiation between normal and pathological bronchi on computed tomography (CT) was studied. The wall thickness of bronchi with normal walls and with pathologically thickened walls were subjectively scored by three independent observers before and after a training period of 2 weeks. Sensitivity, specificity and interobserver agreement were calculated for reading sessions before and after training. Increase and decrease in agreement after training were determined. There was a statistically significant difference (p=0.001) between objectively measured wall thickness of normal and pathological bronchi, both for reference bronchi and for bronchi used for reading sessions. While training increased interobserver agreement, it had no effect on sensitivity (0.46 versus 0.44 after training) and specificity (0.71 versus 0.72 after training) in detecting pathological bronchi. Increased agreement after training was significantly (p=0.001) more frequent than decreased agreement. There is a discrepancy between the effect of training on interobserver agreement and on sensitivity and specificity in the subjective differentiation between normal and pathological bronchi. Interobserver agreement alone is not a reliable indicator of a beneficial effect of training in the evaluation of this parameter.