A Pilot Study of Biomedical Text Comprehension using an Attention-Based Deep Neural Reader: Design and Experimental...

Area of Science:

• Artificial Intelligence
• Biomedical Informatics
• Natural Language Processing

Background:

• Current machine comprehension models excel with general texts but struggle with specialized scientific literature.
• The biomedical domain presents unique challenges due to its expert-level knowledge requirements.
• No existing datasets cater to machine comprehension tasks within biomedical literature.

Purpose of the Study:

• To evaluate the efficacy of deep learning-based machine comprehension models on biomedical articles.
• To develop and validate a large-scale question-answering dataset for biomedical literature.
• To assess the performance of AI models against human comprehension capabilities in this domain.

Main Methods:

• An attention-based deep neural network tailored for biomedical text was developed.
• Pretrained word vectors and biomedical entity type embeddings were utilized to enhance model performance.
• An ensemble method combining multiple models was employed to improve answer accuracy and reduce variance.

Main Results:

• The proposed deep neural network model surpassed baseline models by over 7% on the newly created dataset.
• Human performance evaluation on the dataset revealed the AI model outperformed humans by 22% in comprehension.
• The model demonstrated consistent performance across varying text complexities, unlike human performance which declines with increased difficulty.

Conclusions:

• A novel machine comprehension task and dataset (BioMedical Knowledge Comprehension) were introduced for the biomedical domain.
• The developed deep neural model significantly outperforms human performance in comprehending biomedical literature.
• The AI model's consistent performance highlights its potential for advancing biomedical text analysis and knowledge extraction.