Developing a machine learning model to detect diagnostic uncertainty in clinical documentation.

Area of Science:

• Medical Informatics
• Natural Language Processing
• Machine Learning

Background:

• Unrecognized diagnostic uncertainty can lead to diagnostic errors.
• Studying diagnostic uncertainty is difficult due to a lack of validated identification methods.
• This study aimed to identify linguistic patterns of diagnostic uncertainty in clinical documentation.

Purpose of the Study:

• To identify distinct linguistic patterns associated with diagnostic uncertainty in clinical documentation.
• To develop and validate predictive models for detecting diagnostic uncertainty.

Main Methods:

• A case-control study comparing clinical documentation of hospitalized children with and without an uncertain diagnosis (UD) label.
• Linguistic analysis to identify potential indicators of diagnostic uncertainty, refined by experts.
• Natural language processing (NLP) to categorize medical terms into semantic types.
• Machine learning (ML) models, including random forest, were trained and compared.

Main Results:

• The study included 242 UD-labeled patients and 932 matched controls across 3070 clinical notes.
• The best-performing model, a random forest, achieved 89.4% sensitivity and 96.7% positive predictive value.
• The model utilized a combination of linguistic indicators and semantic types.

Conclusions:

• Combining expert labeling, NLP, and ML with human validation yields highly predictive models for diagnostic uncertainty.
• This approach shows promise for detecting, studying, and mitigating diagnostic uncertainty in clinical practice.
• The findings offer a novel method for improving the accuracy and safety of medical diagnoses.