Improving large language models for adverse drug reactions named entity recognition via error correction prompt...

Area of Science:

• Pharmacovigilance
• Natural Language Processing
• Artificial Intelligence in Medicine

Background:

• Adverse drug reactions (ADRs) monitoring is crucial for patient safety and treatment efficacy.
• Accurate identification of drug names, components, and ADRs via named entity recognition (NER) is vital for drug safety and information integration.
• Existing NER methods struggle with ADRs due to data variability and drug name similarity, often requiring extensive manual annotation.

Purpose of the Study:

• To propose an effective prompt template for ADR entity recognition using large language models (LLMs).
• To enhance the accuracy of identifying drug names, components, and ADRs in complex medical texts.
• To provide a robust method for extracting drug-related entities and building knowledge graphs.

Main Methods:

• Developed a prompt template for ADRs incorporating task descriptions, entity explanations, guidelines, few-shot learning samples, and error correction examples.
• Integrated complex ADR data from the web and created a corpus using the Begin, Inside, Outside (BIO) annotation method.
• Evaluated the prompt template's effectiveness with GPT-3.5 and GPT-4, comparing results against fine-tuned LLaMA and DeepSeek models.

Main Results:

• The proposed prompt template significantly improved GPT-3.5's F1 score from 0.648 to 0.887.
• GPT-4's F1 score increased from 0.757 to 0.921 with the new prompt template.
• The method demonstrated superior performance compared to fine-tuned LLaMA and DeepSeek models.

Conclusions:

• The novel prompt template substantially enhances ADR entity recognition accuracy in LLMs.
• This approach offers a superior alternative to existing NER methods for ADR analysis.
• The findings provide a strong foundation for future drug-related entity relationship extraction and knowledge graph construction.