Infusing Multi-Hop Medical Knowledge Into Smaller Language Models for Biomedical Question Answering

Area of Science:

• Biomedical Natural Language Processing
• Artificial Intelligence in Medicine
• Computational Linguistics

Background:

• Biomedical Question Answering (BQA) tasks, like MedQA-USMLE, demand advanced medical knowledge and reasoning.
• Smaller Language Models (SLMs) struggle with the multi-hop reasoning inherent in complex BQA.
• State-of-the-art large language models demonstrate capability, highlighting a gap for SLMs.

Purpose of the Study:

• To develop a method for enhancing the medical reasoning capabilities of SLMs.
• To bridge the performance gap between large and small language models in BQA.
• To improve SLM performance on tasks requiring multi-hop reasoning.

Main Methods:

• Introduced a multi-hop medical knowledge infusion (MHMKI) procedure.
• Categorized MedQA-USMLE questions by reasoning type and created tailored pre-training instances using Wikipedia data.
• Developed a reasoning chain masked language model for BERT pre-training and a combined QA dataset for GPT fine-tuning.

Main Results:

• MHMKI significantly improved SLM performance across multiple BQA datasets and tasks.
• Accuracy on the MedQA-USMLE dataset increased by an average of 5.3%.
• The procedure particularly benefited tasks demanding multi-hop reasoning.

Conclusions:

• The MHMKI procedure effectively endows SLMs with crucial medical reasoning abilities.
• This approach offers a viable strategy for improving SLM performance in specialized domains like medicine.
• MHMKI represents a significant step towards making advanced BQA accessible with smaller models.