A Comprehensive Natural Language Processing Pipeline for the Chronic Lupus Disease

Area of Science:

• Medical Informatics
• Computational Linguistics
• Artificial Intelligence in Healthcare

Background:

• Electronic Health Records (EHRs) contain vast unstructured patient data, posing challenges for clinical decision-making.
• Extracting specific clinical information like therapies, diagnoses, and symptoms from EHRs is essential for managing chronic diseases such as Lupus.

Purpose of the Study:

• To develop and evaluate a Natural Language Processing (NLP) pipeline for automated extraction of therapies, diagnoses, and symptoms from ambulatory EHRs of Lupus patients.
• To enhance text preprocessing and automate rule identification by integrating rule-based systems, text segmentation, transformer-based topic analysis, and clinical ontologies.

Main Methods:

• A comprehensive NLP pipeline combining rule-based systems, text segmentation, transformer-based topic analysis, and clinical ontologies was developed.
• The approach was applied to a dataset of 750 Italian-language EHRs from 56 patients with chronic Lupus disease.

Main Results:

• The NLP pipeline achieved high performance, with Accuracy and F-score exceeding 97% and 90% respectively across the three extracted domains (therapies, diagnosis, symptoms).
• The integrated approach demonstrated effective enhancement of text preprocessing and automated rule identification.

Conclusions:

• The developed NLP approach significantly automates information extraction from EHRs for chronic Lupus disease.
• This method has the potential for integration into EHR systems to minimize human intervention and support personalized digital health solutions.