REFLECTIVE-TIAB: cost-effective prompt optimization for large language model-based title and abstract screening in...

Title and abstract screening is a labor-intensive stage of systematic reviews. Large language models (LLMs) can automate this process, but performance depends heavily on prompt design and model selection, which is typically manual and time-consuming. Our objective was to evaluate whether automated, reflection-driven prompt optimization improves LLM performance during title and abstract screening.

REFLECTIVE-TIAB uses the GEPA reflective prompt optimizer to improve prompts under an asymmetric loss penalizing false negatives. Nine LLMs screened 8,520 de-duplicated records from a COPD exacerbation predictor search. A 100-abstract gold standard was constructed from inter-model disagreements and was expert-labeled. The prompt was optimized on Llama 3.3 70B via DSPy/GEPA and evaluated across all nine models.

Optimization improved recall across all LLMs (+3.7% to +37.1%). Gemini 3 Flash Preview achieved the highest performance (91% accuracy, F1 81.6%) while costing 25-fold less per abstract than GPT-5.2, which ranked among the lowest-performing models. A prompt optimized on a single open-source model is generalized to all nine without retraining. Total optimization cost was $6.36.

REFLECTIVE-TIAB provides automated, model-transferable prompt optimization for literature screening at negligible cost. Model price did not predict screening performance. The framework could substantially reduce screening workload while preserving comprehensiveness.