Comparison of two major emergency department-based free-text chief-complaint coding systems

Emergency departments (EDs) using free-text chief-complaint data for syndromic surveillance face a unique challenge because a complaint might be described and coded in multiple ways.

Two major ED-based free-text chief-complaint coding systems were compared for agreement between free-text interpretation and syndrome coding.

Chief-complaint data from 21,736 patients at an urban ED were processed through both the New York City Department of Health and Mental Hygiene (DOHMH) syndrome coding system as modified by the Chicago Department of Public Health and the Real-Time Outbreak Detection System Complaint Coder (CoCo, version 2.1, University of Pittsburgh). To account for differences in each system's specified syndromes, relevant syndromes from the DOHMH system were collapsed into the corresponding CoCo categories so that a descriptive comparison could be made. DOHMH classifications were combined to match existing CoCo categories as follows: 1) vomit+diarrhea = Gastrointestinal; 2) cold+respiratory+asthma = Respiratory; 3) fevflu = Constitutional; 4) rash = Rash; 5) sepsis+other = Other, 6) unknown = Unknown.

Overall agreement between DOHMH and CoCo syndrome coding was optimal (0.614 kappa). However, agreement between individual syndromes varied substantially. Rash and Respiratory had the highest agreement (0.711 and 0.594 kappa, respectively). Other and Constitutional had an intermediate level of agreement (0.453 and 0.419 kappa, respectively), but less than optimal agreement was identified for Gastrointestinal and Unknown (0.270 and 0.002 kappa, respectively).

Although this analysis revealed optimal overall agreement between the two systems evaluated, substantial differences in classification schemes existed, highlighting the need for a consensus regarding chief-complaint classification.