Entity-enhanced BERT for medical specialty prediction based on clinical questionnaire data

Area of Science:

• Artificial Intelligence
• Natural Language Processing
• Medical Informatics

Background:

• Medical specialty prediction systems aid remote diagnosis by reducing costs for patients visiting incorrect departments.
• Clinical predictive models using real medical text data face challenges with long sequences and effective integration of domain-specific knowledge.
• Existing methods for extracting entities from clinical text are insufficient for effective model injection.

Purpose of the Study:

• To propose Entity-enhanced BERT (E-BERT), a novel model for medical specialty prediction.
• To effectively inject domain-specific knowledge and focus on relationships between medical entities within clinical text.
• To improve the accuracy and efficiency of medical specialty prediction for first-visit patients.

Main Methods:

• Developed E-BERT, incorporating an entity embedding layer and entity-aware attention mechanism.
• Utilized BERT's structural attributes to enhance the injection of domain-specific knowledge.
• Applied the model to clinical questionnaire data and evaluated its performance against benchmark models.

Main Results:

• E-BERT demonstrated superior performance compared to other benchmark models across various input sequence lengths.
• Visualization of entity-aware attention confirmed effective incorporation of domain-specific knowledge and contextual information.
• The model's robustness and applicability were validated by testing on other Pre-trained Language Models.

Conclusions:

• E-BERT offers an effective solution for medical specialty prediction, leveraging domain-specific entities.
• The proposed method streamlines the diagnostic process and enhances the quality of medical consultations for patients.
• E-BERT provides practical information to first-visit patients, guiding them to the appropriate hospital department.