Interrater agreement in the clinical evaluation of acute pediatric asthma

Prior studies of observer agreement of the clinical exam of children with asthma have focused on small numbers of specially trained observers, often in the setting of clinical trials. Our objective was to evaluate interobserver reliability in the physical exam of acute pediatric wheezing and asthma among a large group of diverse examiners, in a setting of routine clinical practice, and without prior special training. The setting was a large urban children's hospital. Observers were attending pediatric emergency physicians and fellows; hospital respiratory therapists; and emergency department (ED) nurses. Patients were children receiving nebulized medications for wheezing in the ED or inpatient asthma unit. Pairs of observers simultaneously but independently rated work of breathing, wheeze, decreased air entry, prolonged expiration, breathlessness, respiratory rate, mental status, and global (or overall) severity using a structured exam template. A total score for each exam was also evaluated. A total of 230 pairs of observations were performed; mean patient age was 5.3 years. For all pairs, the weighted kappa statistics for the exam components ranged from 0.61 to 0.74 (moderate or substantial agreement). The global severity category and total score had weighted kappas of 0.80 and 0.82, respectively (excellent agreement). Agreement was generally somewhat lower for unlike (different profession) observer pairs than for like observer pairs, but remained acceptable. Agreement in two age groups (< or = 3 years old and > or = 4) was at least moderate for all exam components analyzed. Spearman rank correlations between individual exam components and the global assessments of patient severity were all greater than 0.5, indicating at least moderate to good correlations. We found substantial interobserver agreement among a broad range of examiners in the components of the clinical examination of acute wheezing in both younger and older children. This is contrary to the commonly held observation that the poor interobserver reliability of physical exam findings in asthma may limit their usefulness as asthma outcome measures. Support for use of a structured respiratory exam format or template in asthma guidelines was also shown.