Interactive active learning for literature screening: finetuning GPT with DeepSeek reasoning for cross-domain...

Area of Science:

• Biomedical Informatics
• Artificial Intelligence in Medicine

Background:

• Automated literature screening in biomedicine faces challenges from domain shifts and limited labeled data.
• Large language models (LLMs) struggle with complex, domain-specific reasoning in zero-shot settings.

Purpose of the Study:

• To investigate an interactive, weakly supervised learning framework combining GPT's fine-tuning with DeepSeek's reasoning for improved biomedical literature screening.
• To enhance model accuracy and generalizability across diverse biomedical domains.

Main Methods:

• Developed an active learning framework using model disagreement between GPT-4o and DeepSeek to identify misclassified articles.
• Fine-tuned three GPT variants (GPT-4o, GPT-4o-mini, GPT-4.1-nano) using disagreement-based samples and DeepSeek's rationale traces for weak supervision.
• Evaluated performance on independent benchmark sets in cancer immunotherapy and LLMs in medicine, prioritizing recall.

Main Results:

• Fine-tuning GPT models with disagreement-based examples significantly improved performance.
• GPT-4o-mini achieved the highest F1 score (0.93) and recall (0.95) after fine-tuning.
• Fine-tuned models consistently outperformed zero-shot counterparts across biomedical topics without increasing reviewer workload.

Conclusions:

• Disagreement-driven active learning effectively enhances GPT-based biomedical literature screening.
• Lightweight models like GPT-4o-mini show significant benefits from targeted, reasoning-enriched training.
• The framework offers a scalable solution for efficient and reliable information retrieval in systematic reviews.