Improving large language models for miRNA information extraction via prompt engineering.

Area of Science:

• Biomedical informatics
• Computational biology
• Artificial intelligence in healthcare

Background:

• Large language models (LLMs) offer potential for biomedical knowledge discovery.
• Extracting fine-grained biological information, such as microRNAs (miRNAs), is crucial for understanding disease mechanisms and identifying biomarkers.
• The performance of LLMs in miRNA information extraction requires comprehensive evaluation.

Purpose of the Study:

• To evaluate the capabilities of LLMs in miRNA information extraction.
• To assess the impact of diverse prompt learning strategies on LLM performance.
• To benchmark LLM performance against traditional methods in miRNA data extraction.

Main Methods:

• Construction of three high-quality miRNA information extraction datasets (Re-Tex, Re-miR, miR-Cancer) for benchmarking and training.
• Evaluation of three LLMs (GPT-4o, Gemini, Claude) using baseline, 5-shot Chain of Thought, and generated knowledge prompts.
• Comparison of LLM performance with traditional computational models.

Main Results:

• Optimized prompt strategies significantly improved entity extraction performance.
• Generated knowledge prompting yielded the highest F1 scores (76.6% for entity, 54.8% for relationship extraction).
• GPT-4o outperformed Gemini and Claude; miRNA entity recognition was highest, gene/protein lowest; LLMs did not surpass traditional methods.

Conclusions:

• High-quality miRNA datasets were established for information extraction and knowledge discovery.
• LLM performance in miRNA extraction remains limited, but prompt optimization enhances capabilities.
• Further LLM refinement is necessary to accelerate the discovery of diagnostic and therapeutic targets.