Natural Language Processing to Extract Head and Neck Cancer Data From Unstructured Electronic Health Records.

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare
• Oncology Data Management

Background:

• Electronic Health Records (EHRs) often contain unstructured patient data requiring manual curation.
• Natural Language Processing (NLP) offers a potential solution for rapidly extracting and structuring this data.
• Head and Neck Cancer (HNC) research can benefit from enriched datasets derived from EHRs.

Purpose of the Study:

• To evaluate the effectiveness of an open-source healthcare NLP tool (CogStack) for extracting Head and Neck Cancer (HNC) patient data from unstructured EHRs.
• To assess the performance of the NLP tool after limited supervised training cycles.
• To determine if a thresholding strategy can improve data extraction precision.

Main Methods:

• CogStack was employed to extract SNOMED-CT concepts from HNC patient documents.
• Initial performance was assessed against manually curated ground truth data.
• The model underwent two cycles of supervised training using annotated clinical documents.
• A thresholding approach was implemented to enhance precision, and the final model was evaluated on an unseen test cohort.
• F1 score was utilized as the primary evaluation metric.

Main Results:

• Pre-training, F1 scores were incalculable for 19.5% of concepts due to low recall.
• After one training cycle, F1 scores became calculable for all concepts (median 0.692).
• The final model achieved a median F1 score of 0.708 after further training.
• The test cohort yielded a median F1 score of 0.750, improving to 0.778 with concept-specific thresholding.
• 50 out of 109 SNOMED-CT concepts met predefined criteria for adequate fine-tuning.

Conclusions:

• NLP tools like CogStack can effectively mine unstructured cancer data with limited training.
• Model performance generalized well to an unseen test cohort, indicating robustness.
• A concept-specific thresholding strategy significantly improved extraction performance.
• While generally effective, certain concepts like histopathology terms remained challenging to retrieve accurately.
• The validated NLP approach was successfully applied to extract data for 50 concepts across the entire retrospective HNC cohort.