Development and Evaluation of a Natural Language Processing System for Curating a Trans-Thoracic Echocardiogram (TTE)...

Area of Science:

• Cardiology
• Medical Informatics
• Natural Language Processing

Background:

• Electronic health records (EHR) contain valuable patient data but often rely on unstructured text, posing analysis challenges.
• Echocardiography reports, rich in cardiovascular pathology information, are difficult to extract and analyze due to their narrative structure.
• Natural Language Processing (NLP) is underutilized in echocardiography analysis despite its success in other medical fields.

Purpose of the Study:

• To develop an NLP-based method for extracting and categorizing data from echocardiography reports.
• To convert continuous and discrete outcomes from semi-structured narrative formats into a structured, categorized format.
• To enable future research and clinical applications using structured echocardiography data.

Main Methods:

• A dataset of 135,062 Trans-Thoracic Echocardiogram (TTE) reports was utilized, divided into training, validation, and testing cohorts.
• An NLP system was developed and refined iteratively with medical expert input.
• The NLP system's performance was validated against blind annotations by two clinicians using agreement, discrimination, accuracy, and calibration metrics.

Main Results:

• Continuous outcomes (LVOT VTI, AV VTI, TR Vmax) showed perfect inter-rater reliability (ICC=1.00) and high R² values, indicating excellent NLP-clinician alignment.
• Good inter-rater reliability (ICC=0.75-0.9) was achieved for other outcomes like LVOT Diameter and Ascending Aorta diameter.
• The NLP system achieved a 91.38% accuracy rate for discrete outcome measures in confusion matrix analysis.

Conclusions:

• The NLP-based approach effectively extracts and categorizes data from echocardiography reports.
• The system demonstrated high agreement and concordance with clinician extractions, validating its utility.
• This tool facilitates the conversion of semi-structured echo reports into structured data for improved data management, research, and clinical decision-making.