Performance of a diagnostic system (Iliad) as a tool for quality assurance

Quality Assurance improves health care through detection of quality problems and feedback to the care giver. Current review procedures employed by the Peer Review Organizations (PROs), however, appear to underdetect quality problems, particularly those arising from diagnostic errors. We studied the use of an expert diagnostic system, Iliad, to detect quality problems arising from diagnostic errors. 100 cases were selected from among those Medicare cases reviewed by the Utah PRO (UPRO) and which contained diagnoses recognized by Iliad. Iliad flagged 28 cases out of the 100 as containing diagnostic errors, and a gold standard physician review confirmed quality problems in 17 cases (60.7%). The UPRO review found 28 cases with quality problems, mostly treatment and documentation errors. The quality problems detected by Iliad appeared to be more serious than those detected by the UPRO review. Among the six cases with quality problems detected by both the UPRO and Iliad review, there was none for which the same quality problem was detected by the two procedures. The two review procedures were therefore complementary.