Extracting Knowledge from Scientific Texts on Patient-Derived Cancer Models Using Large Language Models: Algorithm...

Area of Science:

• Biomedical Informatics
• Artificial Intelligence in Oncology

Background:

• Patient-derived cancer models (PDCMs) are crucial for cancer research and preclinical studies, with a notable increase in publications.
• Artificial Intelligence (AI), especially Large Language Models (LLMs), offers potential for large-scale knowledge extraction from scientific literature.

Purpose of the Study:

• To investigate the efficacy of LLM-based systems for automated extraction of PDCM-related entities from scientific texts.
• To compare direct prompting and soft prompting methods using state-of-the-art LLMs.

Main Methods:

• Evaluated direct prompting (manual prompts with instructions, definitions, examples) and soft prompting (automatically trained continuous vector prompts).
• Utilized proprietary GPT4-o and open LLaMA3 family models for experiments.

Main Results:

• GPT4-o with direct prompts achieved competitive results.
• Soft prompting significantly enhanced smaller open LLMs, yielding performance comparable to proprietary models.
• Demonstrated LLMs' potential for domain-specific text extraction.

Conclusions:

• Soft prompting is an effective technique for improving the performance of smaller LLMs in PDCM entity extraction.
• Tailoring LLM approaches to specific tasks and model characteristics is crucial for optimal results in scientific text mining.