Exploration of Genetic Entity Extraction From Spanish Literature Using Generative LLMs.

Area of Science:

• Biomedical Informatics
• Natural Language Processing
• Genomics

Background:

• Organizing gene, genetic variant, and disease information from literature is crucial for precision medicine.
• The vastness of scientific literature necessitates automated information extraction strategies.
• Generative large language models (LLMs) offer a potential solution for automating this extraction process.

Purpose of the Study:

• To systematically evaluate LLM performance in extracting genetic variation-disease impact information from biomedical literature.
• To assess challenges in genetic and disease named entity recognition (NER) in Spanish abstracts.
• To explore various prompting strategies for optimizing LLM extraction accuracy and reducing hallucinations.

Main Methods:

• Utilized the GenoVarDis dataset for evaluating LLMs on Spanish scientific abstracts.
• Experimented with zero-shot, few-shot, and cross-linguistic prompting strategies.
• Investigated the impact of annotation guidelines and output format variations on LLM performance.
• Assessed methods to reduce LLM-generated hallucinations, including providing examples and output structure overviews.

Main Results:

• Prompting language had minimal impact on NER performance, confirming cross-linguistic extraction feasibility.
• Few-shot prompting generally yielded optimal results for information extraction.
• LLMs exhibited a tendency to over-generate or fabricate (hallucinate) entities not present in the source text.
• Providing examples and output structure guidance in prompts significantly reduced hallucinated entities.

Conclusions:

• LLMs show potential for biomedical literature information extraction, particularly with few-shot prompting and grounding techniques.
• While LLMs do not yet match task-specific model accuracy, effective prompting strategies can enhance their utility.
• Grounding LLM outputs in original texts is vital for ensuring accuracy and mitigating hallucinations in biomedical data extraction.