Deep scaled dot-product attention based domain adaptation model for biomedical question answering

Area of Science:

• Biomedical informatics
• Natural Language Processing
• Machine Learning

Background:

• The rapid growth of biomedical literature necessitates advanced text mining techniques.
• Deep learning models, while powerful, often require substantial training data, posing a challenge for specialized domains like biomedicine.
• Existing biomedical question answering systems struggle with limited annotated datasets.

Purpose of the Study:

• To propose a novel hierarchical attention-based transfer learning model for biomedical question answering.
• To address the challenge of insufficient training data in the biomedical domain.
• To leverage BERT (Bidirectional Encoder Representation Transformers) for enhanced semantic representation and domain adaptation.

Main Methods:

• A hierarchical attention-based transfer learning model incorporating BERT for semantic enrichment.
• Utilizing scaled dot-product attention to capture question-passage interactions.
• Employing domain adaptation through fine-tuning to preserve source domain knowledge and improve performance.

Main Results:

• The proposed model achieved state-of-the-art performance on the BioASQ-Task B dataset.
• The system demonstrated superior performance without relying on handcrafted features.
• Outperformed previous best solutions for factoid questions in BioASQ-Task B (2016 and 2017).

Conclusions:

• The developed transfer learning model effectively handles data scarcity in biomedical question answering.
• BERT integration and hierarchical attention significantly enhance model performance.
• The approach represents a significant advancement in automated biomedical information retrieval.