Interrater reliability of clinical findings in children with possible appendicitis

Our objective was to determine the interrater reliability of clinical history and physical examination findings in children undergoing evaluation for possible appendicitis in a large, multicenter cohort.

We conducted a prospective, multicenter, cross-sectional study of children aged 3-18 years with possible appendicitis. Two clinicians independently evaluated patients and completed structured case report forms within 60 minutes of each other and without knowing the results of diagnostic imaging. We calculated raw agreement and assessed reliability by using the unweighted Cohen κ statistic with 2-sided 95% confidence intervals.

A total of 811 patients had 2 assessments completed, and 599 (74%) had 2 assessments completed within 60 minutes. Seventy-five percent of paired assessments were completed by pediatric emergency physicians. Raw agreement ranged from 64.9% to 92.3% for history variables and 4 of 6 variables had moderate interrater reliability (κ > .4). The highest κ values were noted for duration of pain (κ = .56 [95% confidence intervals .51-.61]) and history of emesis (.84 [.80-.89]). For physical examination variables, raw agreement ranged from 60.9% to 98.7%, with 4 of 8 variables exhibiting moderate reliability. Among physical examination variables, the highest κ values were noted for abdominal pain with walking, jumping, or coughing (.54 [.45-.63]) and presence of any abdominal tenderness on examination (.49 [.19-.80]).

Interrater reliability of patient history and physical examination variables was generally fair to moderate. Those variables with higher interrater reliability are more appropriate for inclusion in clinical prediction rules in children with possible appendicitis.