Deep active learning for classifying cancer pathology reports.

Area of Science:

• Computational linguistics
• Machine learning in healthcare
• Bioinformatics

Background:

• Automated text classification is crucial in clinical settings.
• Acquiring labeled data for machine learning and deep learning is costly and difficult.
• Active learning (AL) can reduce the amount of labeled data needed for effective model training.

Purpose of the Study:

• To analyze the effectiveness of 11 active learning algorithms for classifying cancer pathology reports.
• To compare AL strategies using a Convolutional Neural Network (CNN) text classification model.
• To evaluate performance on different dataset sizes and classification tasks (subsite and histology).

Main Methods:

• Implemented 11 active learning algorithms.
• Utilized a Convolutional Neural Network (CNN) for text classification.
• Evaluated performance on small (1K initial samples) and large (15K initial samples) datasets with iterative data addition.
• Compared against random sampling (no active learning).

Main Results:

• All AL strategies, except diversity sampling, outperformed random sampling across tasks and dataset sizes.
• On smaller datasets, marginal and ratio uncertainty sampling showed superior performance.
• Active learning significantly improved performance on rare classes by focusing on underrepresented data.
• No single AL strategy clearly dominated on the large dataset.

Conclusions:

• Active learning reduces annotation costs by guiding efficient sample selection for human annotators.
• Datasets built with effective active learning require less than half the labeled data to achieve similar performance compared to random sampling.
• Active learning is a valuable technique for improving the efficiency of clinical text classification model development.