Detecting clinically relevant new information in clinical notes across specialties and settings

Area of Science:

• Medical Informatics
• Natural Language Processing
• Clinical Data Analysis

Background:

• Automated identification of new vs. redundant information in electronic health records (EHRs) aids clinicians and researchers.
• Evaluating methods to automatically detect clinically relevant new information and compare redundancy across settings is crucial.

Purpose of the Study:

• To evaluate statistical language models augmented with semantic similarity for detecting and quantifying new and redundant information in longitudinal clinical notes.
• To compare redundancy levels across different medical specialties and clinical settings.

Main Methods:

• Utilized statistical language models and semantic similarity measures to analyze clinical notes.
• Generated a reference standard by physician annotation of 591 progress notes from 40 inpatient admissions.
• Evaluated note redundancy across 71,021 outpatient and 64,695 inpatient notes from 500 solid organ transplant patients.

Main Results:

• The best method achieved 0.87 recall, 0.62 precision, and 0.72 F-measure.
• Semantic similarity metrics improved recall but did not significantly alter overall performance compared to baseline.
• Note redundancy was high and similar in outpatient (61%) and inpatient (68%) settings, but varied by specialty (Pediatrics 75%, Internal Medicine 66%, Psychiatry 57%, Surgery 55%).

Conclusions:

• Automated techniques using statistical language models for detecting new vs. redundant clinical note information do not benefit from added semantic similarity measures.
• While redundancy is similar in inpatient and ambulatory settings, it significantly varies across different medical specialties.