Using a Longformer Large Language Model for Segmenting Unstructured Cancer Pathology Reports.

Area of Science:

• Natural Language Processing (NLP)
• Computational Pathology
• Machine Learning for Healthcare

Background:

• Text preprocessing enhances NLP model performance by removing extraneous information.
• Text segmentation isolates key document sections, aiding downstream analysis.
• Transformer models like BERT excel at NLP tasks but have token limitations for lengthy documents.

Purpose of the Study:

• To develop and evaluate a Longformer model for segmenting cancer pathology reports.
• To address the limitations of standard transformer models in processing long documents.
• To improve the accuracy of isolating critical sections within pathology reports.

Main Methods:

• A Longformer Question-Answer (QA) model was fine-tuned on 504 annotated pathology reports.
• The model was trained to identify sections including diagnosis, addenda, and clinical history.
• Performance was compared against baseline methods like regular expressions and BERT QA, using sequence recall, precision, and F1 score.

Main Results:

• The fine-tuned Longformer model achieved an overall sequence F1 score of 0.68 on a test set of 304 reports.
• Specific section F1 scores included diagnosis (0.77), addenda (0.48), and clinical history (0.89).
• The Longformer model demonstrated superior performance in segmenting pathology report sections compared to baseline methods.

Conclusions:

• A fine-tuned Longformer model effectively segments cancer pathology reports.
• This approach enhances the accuracy of isolating key sections for further analysis.
• The developed model offers a valuable tool for computational pathology and NLP applications.