A randomized controlled trial of the accuracy of clinical record retrieval using SNOMED-RT as compared with ICD9-CM

Concept-based Indexing is purported to provide more granular data representation for clinical records.1,2 This implies that a detailed clinical terminology should be able to provide improved access to clinical records. To date there is no data to show that a clinical reference terminology is superior to a precoordinated terminology in its ability to provide access to the clinical record. Today, ICD9-CM is the most commonly used method of retrieving clinical records.

In this study, we compare the sensitivity, specificity, positive likelihood ratio, positive predictive value and accuracy of SNOMED-RT vs. ICD9-CM in retrieving ten diagnoses from a random sample of 2,022 episodes of care.

We randomly selected 1,014 episodes of care from the inpatient setting and 1,008 episodes of care from the outpatient setting. Each record had associated with it, the free text final diagnoses from the Master Sheet Index at the Mayo Clinic and the ICD9-CM codes used to bill for the encounters within the episode of care. The free text diagnoses were coded by two expert indexers (disagreements were addressed by a Staff Clinician) as to whether queries regarding one of 5 common or 5 uncommon diagnoses should return this encounter. The free text entries were automatically coded using the Mayo Vocabulary Processor. Each of the ten diagnoses was exploded in both SNOMED-RT and ICD9-CM and using these entry points, a retrieval set was generated from the underlying corpus of records. Each retrieval set was compared with the Gold Standard created by the expert indexers.

SNOMED-RT produced significantly greater specificity in its retrieval sets (99.8% vs. 98.3%, p<0.001 McNemar Test). The positive likelihood ratios were significantly better for SNOMED-RT retrieval sets (264.9 vs. 33.8, p<0.001 McNemar Test). The positive predictive value of a SNOMED-RT retrieval was also significantly better than ICD9-CM (92.9% vs. 62.4%, p<0.001 McNemar Test). The accuracy defined as 1 (the total error rate (FP+FN) / Total # episodes queried (20,220)) was significantly greater for SNOMED-RT (98.2% vs. 96.8%, p=0.002 McNemar Test). Interestingly, the sensitivity of the SNOMED-RT generated retrieval set was not significantly different from ICD9-CM, but there was a trend toward significance (60.4% vs. 57.6%, p=0.067 McNemar Test). However, if we examine only the outpatient practice SNOMED-RT produced a more sensitive retrieval set than ICD9-CM (54.8% vs. 46.4%, p=0.002 McNemar Test).

Our data clearly shows that information regarding both common and rare disorders is more accurately identified with automated SNOMED-RT indexing using the Mayo Vocabulary Processor than it is with traditional hand picked constellations of codes using ICD9-CM. SNOMED-RT provided more sensitive retrievals of outpatient episodes of care than ICD9-CM.