Accelerated training of bootstrap aggregation-based deep information extraction systems from cancer pathology reports

Area of Science:

• Machine Learning
• Computational Biology
• Deep Learning

Background:

• Bootstrap aggregation (bagging) enhances machine learning task performance.
• Bagging deep neural networks demands significant computational resources and time.
• A key challenge is to improve performance and reduce training time for deep learning models.

Purpose of the Study:

• To investigate if dividing problems into sub-problems and applying partitioned data training can enhance task performance and accelerate training.
• To evaluate the effectiveness of partitioned bagging strategies for deep neural networks.

Main Methods:

• Utilized free text from electronic cancer pathology reports.
• Applied partitioned data training with Multi-Task Convolutional Neural Network (MT-CNN) and Multi-Task Hierarchical Convolutional Attention Network (MT-HCAN) classifiers.
• Split data into 20 sub-problems, performed 2,000 bootstrap resamples, and trained up to 40,000 models concurrently on a high-performance computing environment.

Main Results:

• Aggregated models showed improved task performance compared to single-model approaches.
• Partitioned bagging methods achieved higher classification accuracy on four tasks, notably for cancer histology data extraction (over 500 labels).
• The methods did not yield superior scores for site and subsite classification, indicating partition accuracy depends on data division strategy.

Conclusions:

• The data partitioning and bagging strategy successfully achieved higher performance scores.
• Faster training was realized by leveraging a high-performance supercomputer.
• Partitioned bagging shows promise for complex classification tasks in cancer pathology.