Evaluating NLP Approaches to Extract Drug Indications

Reliable drug-indication knowledge is essential for clinical decision support and pharmacovigilance, yet manual curation is labor-intensive and difficult to scale. This study evaluated nine natural language processing approaches to extract therapeutic indications from FDA Structured Product Labels, benchmarking against 1,838 manually curated indication statements from twenty commonly prescribed medications. Methods included dictionary-based matching (QuickUMLS), a biomedical-pretrained transformer (PubMedBERT), and seven large language models spanning general-purpose and medical domain-specialized architectures. General-purpose LLMs achieved the highest performance, with Gemma2 attaining the best F₁-Score (0.568) despite being the smallest model (2B parameters). Contrary to expectations, biomedical-specialized LLMs underperformed general-purpose counterparts, while dictionary-based matching yielded excessive false positives (F₁ = 0.106). Performance differed markedly by drug, with narrow indication profiles yielding near-perfect accuracy and broader or symptom-adjacent indications proving consistently challenging. These findings establish a benchmark for LLM-based indication extraction and highlight opportunities for hybrid pipelines that balance high recall with precision-oriented validation.