A clustering approach for detecting implausible observation values in electronic health records data

Area of Science:

• Medical Informatics
• Data Science
• Machine Learning

Background:

• Identifying implausible clinical observations in Electronic Health Record (EHR) data is challenging using traditional rule-based methods.
• Anomaly/outlier detection offers an alternative algorithmic approach for flagging questionable values in EHRs.

Purpose of the Study:

• To develop and validate an unsupervised clustering-based anomaly detection approach for identifying implausible EHR observations.
• To test the hypotheses that implausible records are sparse and can be identified by sparse clusters.

Main Methods:

• Applied an unsupervised clustering algorithm to EHR laboratory test data (50 tests).
• Tested various clustering specifications and evaluated performance using confusion matrix indices against silver-standard thresholds.
• Compared the clustering approach with conventional anomaly detection (CAD) methods like standard deviation and Mahalanobis distance.

Main Results:

• The clustering approach demonstrated exceptional specificity and high sensitivity in detecting implausible observations.
• Significantly fewer false positive cases were observed with the proposed clustering method compared to CAD approaches.

Conclusions:

• Developed a novel clustering approach for identifying implausible EHR observations, supported by evidence of their sparsity.
• Established silver-standard plausibility thresholds for 50 laboratory tests, aiding future validation.
• The proposed algorithm enhances human decision-making for improved EHR data quality, necessitating a complementary workflow for action initiation.