Leveraging Large Language Models for Cancer Vaccine Adjuvant Name Extraction from Biomedical Literature.

Area of Science:

• Biomedical Informatics
• Vaccinology
• Artificial Intelligence

Background:

• Accurate identification of cancer vaccine adjuvants is crucial for drug development.
• Existing methods for extracting adjuvant names from literature can be labor-intensive.
• Large Language Models (LLMs) offer potential for automating this process.

Purpose of the Study:

• To evaluate the performance of various LLMs in recognizing cancer vaccine adjuvant names.
• To compare LLM performance against established biomedical text mining models.
• To assess the impact of prompt engineering and few-shot learning on extraction accuracy.

Main Methods:

• Utilized Generative Pretrained Transformers (GPT), Llama, and Gemma models.
• Employed zero- and few-shot learning paradigms on AdjuvareDB and Vaccine Adjuvant Compendium (VAC) datasets.
• Designed specific prompts to extract adjuvant names and evaluate contextual influences.

Main Results:

• GPT-4o achieved high performance (Precision: 65.9%, Recall: 79.7%, F1: 69.8%) on AdjuvareDB.
• Llama-3.2 3B demonstrated strong recall (up to 72.5% with validation) on the VAC dataset.
• All tested LLMs outperformed BioBERT in adjuvant name extraction tasks.

Conclusions:

• General-purpose LLMs show significant potential for automated vaccine adjuvant name extraction.
• LLM performance can be enhanced through prompt design and few-shot learning.
• These findings contribute to advancing vaccine research through improved data mining.