Extracting Biomedical Factual Knowledge Using Pretrained Language Model and Electronic Health Record Context

Area of Science:

• Biomedical Informatics
• Natural Language Processing
• Artificial Intelligence

Background:

• Language Models (LMs) show promise in biomedical natural language processing.
• Current prompting methods for knowledge extraction from LMs have limitations, especially for biomedical knowledge bases (KBs).
• Existing methods provide a low bound for knowledge extraction, insufficient for specialized domains.

Purpose of the Study:

• To enhance knowledge extraction from LMs for biomedical applications.
• To improve the performance of LMs as KBs by incorporating domain-specific context.
• To develop a more robust method for evaluating the knowledge capacity of LMs in the biomedical domain.

Main Methods:

• Experiments were conducted using prompt methods to extract knowledge from LMs, creating LMs as KBs.
• Electronic Health Record (EHR) notes were added as context to prompts to address limitations in the biomedical domain.
• A novel task, Dynamic-Context-BioLAMA, was designed and validated through a series of experiments.

Main Results:

• Language models demonstrated an ability to distinguish correct knowledge from noisy information within EHR notes.
• The proposed method significantly improved the performance of LMs as KBs in the biomedical domain.
• The capability of LMs to differentiate knowledge was validated as a new metric for assessing their knowledge content.

Conclusions:

• Incorporating EHR notes as dynamic context enhances the utility of LMs as biomedical knowledge bases.
• The ability of LMs to discern factual information from contextual noise in EHRs is a key finding.
• This approach provides a novel metric for evaluating the knowledge held by language models, crucial for biomedical applications.