Intraobserver and interobserver agreement of the interpretation of pediatric chest radiographs

The objective of this study is to quantify the magnitude of intraobserver and interobserver agreement among physicians for the interpretation of pneumonia on pediatric chest radiographs. Chest radiographs that produced discordant interpretations between the emergency physician and the radiologist's final interpretation were identified for patients aged 1-4 years. From 24 radiographs, eight were randomly selected as study radiographs, and 16 were diversion films. Study participants included two pediatric radiologists, two senior emergency medicine physicians, and two junior fellowship-trained pediatric emergency medicine physicians. Each test included 12 radiographs: the eight study radiographs and four randomly interspersed diversion radiographs, and each radiograph was paired with a written clinical vignette. Testing was repeated on four occasions, separated by > or =2 weeks. The dependent variable was the interpretation of presence or absence of pneumonia; primary analysis done with Cohen's kappa (95% confidence intervals). Intraobserver agreement was good for pediatric radiologists (kappa = 0.87; 95% CI 0.60-0.99) for both but was lower for senior emergency physicians (mean kappa = 0.68; 95% CI 0.40-0.95) and junior pediatric emergency physicians (mean kappa = 0.62; 95% CI 0.35-0.98). Interobserver agreement was fair to moderate overall; between pediatric radiologists, kappa = 0.51 (0.39-0.64); between senior emergency physicians, kappa = 0.55 (0.41-69), and between junior pediatric emergency medicine physicians, kappa = 0.37 (0.25-0.51). Practicing emergency clinicians demonstrate considerable intraobserver and interobserver variability in the interpretation of pneumonia on pediatric chest radiographs.