Classifying unstructured electronic consult messages to understand primary care physician specialty information needs

Area of Science:

• Health Informatics
• Machine Learning in Healthcare
• Natural Language Processing

Background:

• Electronic consultations (eConsults) contain valuable information on clinician needs.
• Extracting this information is challenging.
• Developing automated methods is crucial for efficient data utilization.

Purpose of the Study:

• To develop machine learning models for classifying eConsult questions by type and content.
• To assess the feasibility of reducing expert time for data labeling.
• To explore the effectiveness of multitask learning in this domain.

Main Methods:

• Utilized a large dataset of deidentified eConsults from 2008-2017.
• Developed classifiers using Bidirectional Encoder Representations from Transformers (BERT).
• Experimented with multitask learning and analyzed learning curves to evaluate labeling reduction.

Main Results:

• Multitask learning provided benefits primarily in the neurology-urology pair due to similar question type distributions.
• Continued pretraining of models in new domains proved highly effective.
• Achieved near-peak performance with significantly reduced data (10%) in the neurology-urology pair when combined with neurology data.

Conclusions:

• Accurate classification of eConsult content is achievable with sufficient labeled data.
• Methods for reducing labeling effort are effective only in specific scenarios.
• Further research into novel learning paradigms is needed to minimize labeling requirements.