CiteSure: retrieval-augmented large language models for faithful biomedical citation recommendation

Area of Science:

• Biomedical research
• Artificial intelligence
• Natural language processing

Background:

• Accurate citations are crucial for scientific integrity.
• Large language models (LLMs) often generate inaccurate or fabricated citations.
• Existing retrieval-augmented generation (RAG) methods lack semantic precision in evidence retrieval.

Purpose of the Study:

• Develop a domain-specific RAG system for reliable biomedical citation recommendations.
• Enhance context-specific reference generation using LLMs.
• Improve factual accuracy and relevance in scientific citations.

Main Methods:

• Introduced CiteSure, a sentence-level citation recommendation tool.
• Employed a 2-stage RAG framework with BioLLM2Vec (dense retriever) and BioRankLLaMA (reranker).
• Adapted LLaMA3-8B-Instruct using biomedical-specific data and evaluated on an Alzheimer's disease dataset.

Main Results:

• CiteSure achieved 100% factual accuracy and a 77.50% relevance score.
• BioLLM2Vec demonstrated over 80% accuracy in retrieving relevant articles.
• BioRankLLaMA outperformed baseline rerankers in key metrics (MAP, MRR, Precision@5).

Conclusions:

• CiteSure effectively integrates domain-specific retrieval with LLM generation for improved citation recommendations.
• Domain-specific adaptation is vital for enhancing biomedical citation accuracy.
• Publicly available datasets, models, and code are provided to support future research.