Supervised Text Classification System Detects Fontan Patients in Electronic Records With Higher Accuracy Than ICD...

Area of Science:

• Medical Informatics
• Computational Biology
• Clinical Data Analysis

Background:

• The Fontan operation is linked to significant patient morbidity and mortality.
• Identifying Fontan patients using International Classification of Diseases (ICD) codes is challenging due to limitations in code accuracy.
• Accurate identification of Fontan cases is crucial for research and clinical management.

Purpose of the Study:

• To develop and compare natural language processing (NLP)-based machine learning models for automatic Fontan patient detection.
• To evaluate the performance of NLP models against traditional ICD code-based classification.
• To enhance the creation of large Fontan patient cohorts for research.

Main Methods:

• Utilized free-text clinical notes from 10,935 manually validated patients (7.1% Fontan).
• Trained and optimized machine learning models, including Support Vector Machines (SVM) and RoBERTa (a transformer-based language model), on 80% of the data.
• Implemented a novel sliding window strategy for RoBERTa to manage text length limitations and evaluated models on held-out data using F1 score.

Main Results:

• Support Vector Machines achieved the highest F1 score (0.95), outperforming both RoBERTa (0.89) and ICD code classification (0.81).
• Both NLP models demonstrated significantly higher accuracy than ICD code-based classification (P<0.05).
• The sliding window strategy improved RoBERTa's performance but did not surpass SVM; ICD codes generated more false positives.

Conclusions:

• NLP models can automatically identify Fontan patients from clinical notes with superior accuracy compared to ICD codes.
• These findings highlight the potential of NLP in improving patient cohort identification for Fontan-associated research.
• Further advancements in NLP offer promising avenues for more precise clinical data analysis.