Path-BigBird: An AI-Driven Transformer Approach to Classification of Cancer Pathology Reports.

Area of Science:

• Natural Language Processing
• Computational Pathology
• Oncology

Background:

• Surgical pathology reports are crucial for cancer diagnosis and treatment decisions.
• Accurate and timely extraction of tumor characteristics from these reports is essential for effective cancer management.
• Current methods for information extraction may not fully capture the nuances of pathology data.

Purpose of the Study:

• To explore the impact of domain-specific transformer models on extracting tumor characteristics from cancer pathology reports.
• To develop and evaluate a specialized transformer model, Path-BigBird, for pathology report analysis.
• To compare the performance of Path-BigBird against existing classification and clinical transformer models.

Main Methods:

• Developed Path-BigBird, a transformer model trained on 2.7 million pathology reports from SEER cancer registries.
• Compared Path-BigBird variations with Hierarchical Self-Attention Network (HiSAN) and Clinical BigBird models.
• Evaluated models on five information extraction tasks: site, subsite, laterality, histology, and behavior, using macro and micro F1 scores.

Main Results:

• Path-BigBird and Clinical BigBird outperformed HiSAN across all evaluated tasks.
• Path-BigBird demonstrated superior performance on the challenging subsite and histology extraction tasks.
• Path-BigBird achieved significant performance gains, particularly in histology extraction, improving micro F1 by 1.44 and macro F1 by 3.55 points over HiSAN.

Conclusions:

• The Path-BigBird model enhances automated information extraction from pathology reports.
• Domain-specific models like Path-BigBird offer improved accuracy for complex pathology data.
• Less computationally intensive models like Clinical BigBird and HiSAN retain utility in resource-constrained settings.