Interobserver variation in the chest radiograph component of the lung injury score

The lung injury score is a semi-quantitative system used in the definition and grading of the acute respiratory distress syndrome. It is composed of two, three or four equally weighted components. One component is derived from the chest radiograph, which may contribute up to 50% of the total score. A score of 1 is awarded for each quadrant on the chest radiograph which contains alveolar consolidation. We examined the interobserver variation between two anaesthetists, two radiologists and two critical care physicians who scored blindly 100 chest radiographs from patients with adult respiratory distress syndrome. There was very good agreement between the two radiologists in the total scores (kappa 0.97) and in individual scores in each of the 4 quadrants (kappa 0.97-1.0). The agreement between anaesthetists and radiologists was only fair for the total score (kappa 0.37-0.42), but moderate to good for individual quadrant scores (kappa 0.43-0.73). The agreement between the two anaesthetists was moderate for individual quadrant scores (kappa 0.44-0.60), but only fair for total score (kappa 0.34). There was poor agreement between the two critical care physicians for total score (kappa 0.05) and for individual quadrant scores (kappa 0.04-0.20). Agreement between the physicians and other observers was poor to fair for the total score (kappa 0.12-0.32) and poor to moderate for the individual quadrant scores (kappa 0.15-0.63). Both anaesthetists and physician 2 underestimated the overall chest scores (median scores 2, 3 and 1 respectively) in comparison to the radiologists (median scores 3.5). Physician 1 significantly overscored (median score 4). The chest radiograph component of the lung injury score can be consistently assessed by radiologists, but significant variations may be introduced when assessed by other clinicians. This has significant implications for the use of the lung injury score in studies of adult respiratory distress syndrome and other studies which incorporate radiographic appearances in the definition.