Large Scale Semi-Automated Labeling of Routine Free-Text Clinical Records for Deep Learning.

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare
• Computational Pathology

Background:

• Breast cancer screening mammography reduces mortality but leads to high rates of unnecessary recalls and biopsies.
• Deep learning for mammography requires large-scale labeled datasets, which are challenging to acquire.
• Developing these datasets is hindered by the difficulty and cost of manual data annotation.

Purpose of the Study:

• To develop a hybrid framework combining traditional Natural Language Processing (NLP) and IBM Watson for automated annotation of breast pathology reports.
• To overcome barriers in creating large-scale labeled datasets for deep learning in mammography.
• To compare the performance of traditional NLP techniques with IBM Watson's Natural Language Classifier (NLC).

Main Methods:

• A hybrid framework was developed using traditional NLP techniques and IBM Watson.
• An expert reviewer manually annotated 3521 breast pathology reports.
• Seven machine learning classifiers and IBM Watson's NLC were evaluated using precision, recall, and F-measure.
• Logistic regression was identified as the best performing traditional NLP method.

Main Results:

• Both traditional NLP and IBM Watson achieved high performance (F-measure > 0.96) for reports under 1024 characters.
• Traditional NLP performance decreased for reports over 1024 characters (F-measure = 0.83).
• The hybrid framework successfully annotated over 10,000 breast pathology reports.

Conclusions:

• A hybrid NLP and IBM Watson framework efficiently annotates breast pathology reports, significantly accelerating dataset development for mammography deep learning.
• This approach effectively addresses the challenge of creating large, labeled datasets crucial for advancing AI in breast cancer screening.
• The method shows strong performance, particularly for shorter text cases, offering a scalable solution for medical data annotation.